How to Improve the Performance of xml Library Files

[radiodungog]

Dmitry,

We use several xml music libraries to make searching easier for our live presenters.  Typical examples are:  Main; Top-40 and Country.

When we swap from one xml library to another there is always a significant delay when it goes through the procedure of "Loading base" and "Searching 0% to 100%".  This happens again when we search different fields in "Search In" - "Title"; "Artist"; "Album"; "Year" and so on.

Our main RadioBOSS playout PC has a spinning HDD and that seems to be very slow.  Another PC (my home PC with an SSD and lots of RAM) seems much quicker and has a delay of about 15 seconds.

So I'm guessing that we should upgrade our main studio PC to include an SSD to improve it's performance.  Is there anything else that you would suggest to improve things ?

Most RadioBOSS functions work very quickly and this xml library function seems to be the slowest feature in the software.  In the past we stored the xml library on our LAN connected NAS - but found that to be very very slow - probably due to the large size of the xml library file and the latency of the LAN.  So now we have the xml library files stored locally on each PC.


Thanks,  Jamie C.

 

[djsoft]

Until we add support for more traditional databases (which handle large arrays of data way better than XML files), you can speed up things significantly by using so called RAM drive.

Software:
http://www.ltr-data.se/opencode.html/#ImDisk

How to use:
http://www.tekrevue.com/tip/create-10-gbs-ram-disk-windows/

In short, it's a virtual disk in computer memory (RAM). Read/write operations on such a disk are extremely fast, even compared with SSD. The only downside is that contents of such a disk are erased at each reboot (this can be solved).

 

[radiodungog]

Dmitry,

Ah good.  I used to do this in the old DOS days with XT PCs.  Excellent idea !


Thanks,  Jamie C.

 

[djsoft]

Ah good.  I used to do this in the old DOS days with XT PCs.  Excellent idea !

Very well! I hope it will improve things significantly.

 

[radiodungog]

Dmitry,

I have created a RAM disk for a test and the "Loading base" and "Searching 0% to 100%" now takes about 5 seconds - and flicking from one library to another is very fast.

This is amazing and will certainly solve our problem.  I now need to do it permanently and document it.



Thanks,  Jamie C.

 

[djsoft]

I have created a RAM disk for a test and the "Loading base" and "Searching 0% to 100%" now takes about 5 seconds - and flicking from one library to another is very fast.

This is amazing and will certainly solve our problem.  I now need to do it permanently and document it.

That's great! I think you can set RAM disk creation (and copying library files there) to startup, this way it will be automatic. Depending on how often library files are changes, you can also consider scheduling (in RB's scheduler) periodic copy of library files to RAM disk from your main folder.

 

[pety]

Dmitriy,
I would like (RAM Drive) to be implemented in RadioBOSS.
That will solve "Read additional info" in huge Library too.

 

[djsoft]

Dmitriy,
I would like (RAM Drive) to be implemented in RadioBOSS.
That will solve "Read additional info" in huge Library too.

It can be implemented another way, e.g. RadioBOSS can cache all this data in memory. This will increase memory requirements a lot, but operation will be light speed. As memory is cheap now, I think this can be added in one of the next updates.

 

[pety]

Dmitriy,
I would like (RAM Drive) to be implemented in RadioBOSS.
That will solve "Read additional info" in huge Library too.

It can be implemented another way, e.g. RadioBOSS can cache all this data in memory. This will increase memory requirements a lot, but operation will be light speed. As memory is cheap now, I think this can be added in one of the next updates.

GREAT !

 

[radiodungog]

Dmitry,

We have some music request shows that are broadcast live at Radio Dungog and this is the time that the search function really gets a work out.  And, if they are not careful to wait for the loading and indexing to finish they can lock up the music library and get stuck for the night.

This solution is a good one.  To have something like this incorporated in RadioBOSS as standard would be even better because setting up the RAM drive and the batch files to copy over the content is not an easy job for everyone.


Thanks,  Jamie C.

 

[djsoft]

This solution is a good one.  To have something like this incorporated in RadioBOSS as standard would be even better because setting up the RAM drive and the batch files to copy over the content is not an easy job for everyone.

Yes, an option "cache all libraries" will be added in one of the next updates (5.2.x).

 

[pety]

This solution is a good one.  To have something like this incorporated in RadioBOSS as standard would be even better because setting up the RAM drive and the batch files to copy over the content is not an easy job for everyone.

Yes, an option "cache all libraries" will be added in one of the next updates (5.2.x).

That is awesome !

 

[radiodungog]

Dmitry,

Something quite weird has happened with this RAM drive.  I set it up as the V: drive because V was a sensible choice for "Virtual" drive.  I did all the tests and reported back here with the results and then shut down the PC and turned off the power.

The next day when I started the PC I could see the V drive was still there with all the xml files.  I thought I would have to go through the whole process again because all this was created and stored in RAM.

This has continued the same every day because I shut down the PC every night and turned off the power.

I'm using Windows 8.1 Pro.  What do you think has happened ?  Has Windows cached the RAM contents as it shut down and restored it on bootup ?


Thanks,  Jamie C.

 

[djsoft]

I'm using Windows 8.1 Pro.  What do you think has happened ?  Has Windows cached the RAM contents as it shut down and restored it on bootup ?

It could be that software which you've used to create RAM drive creates it for you automatically. That's very good as it means less work for you

 

[radiodungog]

Dmitry,

Yes, amazing.  I wonder if it created an autoexec.bat file or something similar to get it done at every bootup ?  Whatever it has done certainly makes less work.  Cool !


Thanks,  Jamie C.

 

[pety]

This solution is a good one.  To have something like this incorporated in RadioBOSS as standard would be even better because setting up the RAM drive and the batch files to copy over the content is not an easy job for everyone.

Yes, an option "cache all libraries" will be added in one of the next updates (5.2.x).

Dmitriy, please do not forget about this: "cache all libraries". It will be an awesome option in RadioBOSS. How much RAM do you think that will take some xml libraries? Becouse I think that will not be so much.

 

[djsoft]

This solution is a good one.  To have something like this incorporated in RadioBOSS as standard would be even better because setting up the RAM drive and the batch files to copy over the content is not an easy job for everyone.

Yes, an option "cache all libraries" will be added in one of the next updates (5.2.x).

Dmitriy, please do not forget about this: "cache all libraries". It will be an awesome option in RadioBOSS. How much RAM do you think that will take some xml libraries? Becouse I think that will not be so much.

The size will be approximately a size of an XML library file size. This feature is in the queue to be implemented, approximately in RB 5.2.3 (the 5.2.2 release will mostly target bug fixes).

 

[pety]

Cool !
I will remember you about implementing this option in 5.2.3 

 

[radiodungog]

Pety,

You're a hard taskmaster

I'm setting up the V: drive on our production PC now and will see how that goes for the next week then I'll add it to our main on air PC.  Our xml library files are overdue for some attention and to be regenerated - but I'll get the V: drive running first and regenerate the xml files in a week or two.

All of our xml music library files are about 500MB.


Regards,  Jamie C.

 

[radiodungog]

Dmitry,

Looks like the V: drive needs to be created and re-populated on Windows 7.  I just installed it on a Windows 7 PC in the studio (production PC) and the V: drive disappeared after a reboot.


Thanks,  Jamie C.