P Squared Forum




Myriad 3.5.12 stalling…

Author Message
 Posted 25 May 2010 20:24:37
Supreme Being

Supreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme Being

Group: Forum Members
Last Login: 10 May 2012 11:34:12
Posts: 136, Visits: 620
Myriad 3.5.12 stalling…

I have a randomly appearing problem that has only occurred a few times in the last couple months, and I have only been conscience of it since I installed Myriad 3.5.14 then uninstalled it and reinstalled Myriad 3.5.12.  It appears to be only affecting the instance of Myriad 3.5.12 in our virtual studio PC (Rack room located automated/overnight play-out PC) as this machine physically runs 24hrs a day, but only plays materials out for 10 hours across a our schedule. I’ve not had similar reports from daytime users of either of our two ‘live’ studios and the instances of Myriad therein where the turned off overnight.

What I understand of the problem is that Myriad stalls for no apparent reason at the end of a log item in an ‘Auto-Fade’ log hour. This could be a song, cart, link, VT, advert, anything!  Last night (25/05/10) this happened at 00:52hrs and last Thursday (20/05/10) at 06:05hrs, before these dates there have been a couple of other occurrences overnight usually around 03:00hrs. However, I attributed these to another issue I was fire fighting at the time; which was a Trojan virus in the systems, but that might now be a wrong assumption.  Thankfully the virus didn’t make its way to the Myriad server, only the email/admin server and a laptop and two desktop PC.  (Before anyone says anything about MS security updates downloading at the default 03:00hrs, this option has been suspended on all LAN connected PCs via group policies in SBS2003.)

Does anyone have any ideas?

Roger

Roger Woods Producer/Presenter/Engineer

Post #4138
Add to Twitter Add to Facebook
 Posted 25 May 2010 23:02:39
Supreme Being

Supreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme Being

Group: Forum Members
Last Login: Yesterday @ 19:45:36
Posts: 101, Visits: 898
I run Myriad on two PC's connected on a LAN with a Windows 2003 Server as our DC/AD/DNS/DHCP/Print Server. Myriad audio runs locally.

You've probably done this allready, check the Windows Event viewer on the PC/Server that is running myriad around the hours that the pause is happening.

If it's early hours and appears to be happening alot then you must have something on the network secheduled to do some tasks?? What PC/Server is serving the Audio, local or network? Could be bandwidth, AV Scans (database lock files), Group policy updates, dns leases, AD replication??

If not software then it could be hardware related, i'd then check the Automation card (make sure the ground is connected), Soundcard freezing (this does happen usually a reboot to fix) and last but not least faulty memory.

Just a few ideas for software/hardware but i'm sure you've checked these?

The last un-explained myriad crash/freeze I experienced was to do with a faulty motherboard related to overheating.
Post #4139
Add to Twitter Add to Facebook
 Posted 26 May 2010 16:00:44
Supreme Being

Supreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme Being

Group: P Squared
Last Login: 2 days ago @ 14:26:47
Posts: 1,413, Visits: 2,295
Hi Roger,

How long does playout stop for?


------------
Peter Jarrett, Technical Director
P Squared Ltd.

Bill Bailey: "No win, no fee, no basis in reality. Just a room above a minicab office in Acton and a steady stream of greedy simpletons whose delusion is only matched by their clumsiness"

Post #4146
Add to Twitter Add to Facebook
 Posted 27 May 2010 06:47:34
Supreme Being

Supreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme Being

Group: Forum Members
Last Login: 10 May 2012 11:34:12
Posts: 136, Visits: 620
Hi Peter,

The machine stops until such time the fault is discovered and the PC is rebooted! This is often by the breakfast show presenter...

Sorry, as basic piece of information left our of the detail...

Roger.


Roger Woods Producer/Presenter/Engineer
Post #4151
Add to Twitter Add to Facebook
 Posted 30 May 2010 08:50:04
Supreme Being

Supreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme Being

Group: Forum Members
Last Login: 10 May 2012 11:34:12
Posts: 136, Visits: 620
I don't usually get wound up about minor 'glitches', however, this is starting to take to much of my attention in getting to the bottom of the issue.

The overnight PC, after months of happily running Myriad without fault, is stalling on a daily basis now during 'auto-fade' hours, so much so I'm seriously thinking of having to install a silence detector from Sonifex. We've too much dead-air overnight and this is starting to concern me.

Investigating the Windows event logs, as suggested above, I can see that issues recorded as an error in Myriad 3.5.0.12, etc., etc., in the kernal32.dll, etc., etc.

With this knowledge I trawled the MS knowledge base and the conclusion was that I need to reinstall the kernel32.dll. I tried the quick solution; extracting from the master CD and renaming and inserting it into the system 32 folder... But ended up having to do a full system re-install.

In my hast to do so I forgot one major thing... to un-license Myriad from the PC I rebooted and now I have to wait until Tuesday to get the tech support time to sort things our for me as the automated licensing system won't allow me to re-license; so I'm a PC down, which is frustrating but my own fault.

However, this is not he end of the story... Given the above we''ve had to run our overnights from a studio, and would you believe it the same issue is happening on the machine we used last night. At 01:27 the machine stalled, reported a kernal32 error and required rebooting, this I did at 05:55 this morning, when the bird song got to much for me and I had to get up and make some tea, while waiting for the kettle I switched the radio on and... nothing. So my day started by logging in remotely to our the studio site and forcing life back into to failing system. I'm starting to loose real confidence in 3.5.0.12, I don;t want to install 3.5.0.14 because of the issues it has, which will 'trouble' some of our team if things don't do what they've become dependent upon.

Does, anyone have any constructive suggestions to a way ahead from here?

I have already re-installed 3.5.0.12. The three Myriad PC run the sames applications on top of MS Win XP Pro SP3, these are Myriad 3.5.0.12, Dimension4, LogMeIn, Delta1010 drivers, Advantec drivers, Matrox G500 drivers (except our rack play-out PC - Intel graphics drivers & Echo Mia audio drivers) The PC are apart of our LAN/WAN domain administered by our MS SBS2003 server. The Myraid audio store runs on a dedicated server running MS SBS2003 also.

Roger.



Roger Woods Producer/Presenter/Engineer
Post #4160
Add to Twitter Add to Facebook
 Posted 30 May 2010 17:09:15
Supreme Being

Supreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme Being

Group: Forum Members
Last Login: Yesterday @ 19:45:36
Posts: 101, Visits: 898
Roger,

I have read your post in full. A few more suggestions, as this problem is now on two PC's it could be a bug in Myriad or a Network fault. I would suggest you check the LAN Cabeling and the Switch. I'd replace all cabels with pre-molded patch and replace the swicth. I have had a faulty switch cause similar crashes. The other suggestion is that are permissions being change on the Myriad share ovenight by a backup program??

I would also suggest running a bandwidth network monitor overnight to see if there are any bottle necks in your LAN setup causing this issue??


Many thanks
Shane
Post #4162
Add to Twitter Add to Facebook
 Posted 31 May 2010 09:02:36
Supreme Being

Supreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme Being

Group: Forum Members
Last Login: 10 May 2012 11:34:12
Posts: 136, Visits: 620
Thanks for the input, Shane, it's much appreciated.

Our network is solid-core structured cabling from patch panel in the rack to the sockets on walls in the studios and elsewhere. All 'jumper' cables are molded Cat6 leads, all where new out of their wrapper. The hub is a 3Com 48port intelligent switch 10/100 with 2 x 1000mb (1Gb) ports where the servers are connected and is working faultlessly.

I've checked the application logs again today; we had a stall at 03:32 this morning on another PC running Myriad last night. So that's all of our Myriad machine faulting now... same problem Kernel32.dll in Myriad 3.5.0.12. The servers logs show that there's not been a problem recorded at the time of the stall or in the last 48hrs.

The only thing I can think of is that on all our Myriad PC, and others, I have Dimension4 running. This is a small application that keeps our PC to time, they update every 15 mins, by accessing a remote time server. It could be possible that this software checks for a time update at the same time Myriad is requesting a files from the audio server. But I doubt it however.

Perhaps I'll investigate a server based time update option.

The only backup I have running is on the admin server and this runs daily at 11pm to that days backup HD caddy. This has been running like this for the past 9 months without fault so, it's not a back up issues. Myriad's audio store is a mirrored server, with 2 x 1TB disk, presently there's 400Gb used, so there plenty of space there. I will occasional back this up to an external usb drive.

I could run a anti-virus check, but I did this recently with another issue.

Roger.

I'm running out of ideas... I'll talk to the teck team at P2 tomorrow.


Roger Woods Producer/Presenter/Engineer
Post #4163
Add to Twitter Add to Facebook
 Posted 31 May 2010 11:18:32
Supreme Being

Supreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme Being

Group: Forum Members
Last Login: Yesterday @ 19:45:36
Posts: 101, Visits: 898
Roger,

My last idea is to run a process monitor and capture the data to a log file. This needs to log on a big drive - Sysinternals i think is the one I have used before. This will list all files access/processes/dll's and other usefull information that will help you and Psquared to track down what is causing the crash/stall. You can then look at the time of the crash/stall and filter it from the log and see what is happening on that PC at that time. If you' ve not tried this before I would strongly recommend it.

Many thanks
Shane
Post #4164
Add to Twitter Add to Facebook
 Posted 31 May 2010 14:05:35
Supreme Being

Supreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme Being

Group: Forum Members
Last Login: Yesterday @ 19:45:36
Posts: 101, Visits: 898
Roger,

Sorry to keep posting but another thing that would create the myriad freeze/crash that you are having is the 'autodisconnect' in the registry. From your posts this crash is only overnight when no presenters are doing shows.

In Windows XP Pro/7 Pro there is a reg key for auto-disconnect, this is set usually at 15mins but I have had this issue when our PC's disconnected overnight and this crashed myriad, it just freezed and then the kernal error. Also worth checking the power save on the network cards, this should be turned off but if you have rebuilt the PC's this will be on by default and the same for the regkey.

Many thanks
Shane
Post #4165
Add to Twitter Add to Facebook
 Posted 31 May 2010 18:20:46
Supreme Being

Supreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme BeingSupreme Being

Group: Forum Members
Last Login: 10 May 2012 11:34:12
Posts: 136, Visits: 620
Keep the thoughts coming, Shane! Keep them coming...

I'll look at you're suggestion of logging the entire going's on with the PC, but until I get my third instance fo Myriad running I wait a bit and see transpires.

I managed to grab a few hours at the station today, and tidyed up the Myriad PC's of the detritus that acculmates in odd places - 12 gigs worth of recordings by presenters ROTing their shows and not deleting the files once they're burnt to disc, etc., etc. on the audio server alone. Spotted a couple of PCs with both Dimension4 running as a servers as well as the Windows time server active in Services.

Deleted the presenters roaming folders and turned off the caching to leave these at the server from now on in. This seems to have speeded things up at log-in. Removed a few roaming profiles too from team memebers who've not been around laterly.

Also changed the IP address of the 'OCP' in Myriad to send to the second lan port on the audio server, as I 've been having a few issues with this dropping ocassionally.

As for you're thoughts on auto-disconnect, I've always had a line in the login.bat ('net config server /autodisconnect:-1') but hadn't run it on the two servers, this has now been done. As has the following on the servers too.

All our desktop (& Laptop) PC are HP/Compaqs. The desktops being EVO D510 SFF, they have all the latest BIOS, firmware and drivers. I did however double check that the network adaptors had had 'sleep' and 'power saving' options turn off, not all had. So again I monitor events and see what happens.

Also, finally managed to get our colour laserjet installed and working, it's been sitting around as its was thought to be faulty... But it wasn't it just need it's IP addressing tweaked.  As you may have read elsewhere on the forum all our our IT stuff is second hand, large ex-corporate and three years old - the organisation we recevied it from likes to recycle tthier kit o charitable groups.

Roger.

Roger Woods Producer/Presenter/Engineer

Post #4166
Add to Twitter Add to Facebook


Reading This Topic

Expand / Collapse