Slow Searches with Digibase

[stormtrooperpez]

Hello,

Once again I seek answers to the same perplexing problems: why are searches with Digibase so slow, and why does the search results window take so long to update?

I'm trying to search across 5 drives over a 100BaseT network. The drives contain nearly 100GB of sound effects in aiff and SDII formats.

Using OSX's toolbar search, I type in 'auto', and in about 40 seconds I get 3,841 results. I can scroll through the list, and the screen updates almost as fast as I can move the scroll bar.

Using Digibase, I replicate the search. I choose the 5 drives, type in 'auto' and hit search. I get a message saying, 'searching databases' and after 1 minute and 19 seconds, I get the same number of results (3,841) - but then Digibase starts 'searching the file system' and after 5 minutes and 52 seconds, the search is finally complete (with no more results beyond the first 3,841).

The second problem is with the search results window. I try to move the scroll bar, and I get the spinning beach ball for 30 seconds, and then the screen updates, although not to where I let go of the scroll bar. And forget about auditioning the audio file.

What gives? Is Digibase simply incapable of dealing with large numbers of results? If so, it makes for a pretty 'lame duck' sound effects database solution. Everyone who is currently using PT 6 on OSX (10.2.6) here on my team at EA is having this problem. Unfortunately, we are stuck with it because our old system (Sonic Science's Sonic Search) will probably never be ported to OSX. In OS9, this system was super fast, completing the same search as outlined above in about 5 seconds.

[DigiTechSupt]

Have you indexed those drives? As large as they are it could take some time. Select the drives (maybe one at a time to start) in your Workspace window and choose "Update database for selected" in the toolbox menu.

Jon Connolly
Digidesign Product Specialist

[Paul Greyson]

DigiBase does searches in 2 passes. The first pass searches the databases for the areas being searched. The second pass goes directly to the file system in case the databases are not up to date (e.g. because you've made modifications to the file system via the Finder.) If you've indexed your SFX volumes then you can stop the search once it gets to the "Searching File System" stage.
wrt the performance problems you're seeing, I wonder why it's important to you to be able to browser nearly 4,000 results. I suppose it's a reasonable test if you just want to compare the performance of various search systems, but practically speaking, if your search gets you thousands of results, it seems like you're going to have to refine the search anyway, no?
I'd also be curious to know how many files you have on those 5 volumes. You may be better of generating a number of DigiBase "catalogs" for sections of your SFX library rather than just indexing the network volumes as you've done.

[MusicTrax]

Originally posted by stormtrooperpez: Is Digibase simply incapable of dealing with large numbers of results? If so, it makes for a pretty 'lame duck' sound effects database solution

<font size="2" face="Verdana, Arial">---------------------------------------------------------------


My gut feeling is that Digibase may not have been designed for heavy-duty sound designers who have hundreds of gigabytes of many thousands of files.

For that kind of application, I think you need a dedicated tool like Soundminer, which can run in the background with ProTools -- even under OSX -- and let you audition and import sounds very quickly into ProTools sessions. You can get more info at Soundminer.com.

--Marc W.

[stormtrooperpez]

Thanks for the replies, everyone.

Paul, you were right when you said that 4000 results isn't a very practical search, I was just doing a comparison. The problem is that I'm seeing the same sluggish screen redraws with as little as 50 results, but only with the networked drives. However, doing the OSX toolbar search, or using a program like Soundminer, the results window is totally responsive - so obviously this is not a problem with my network speed, but rather a problem with how Digibase is reading those drives.

Thinking along those lines, I think I may have found the source of the problem. If I move every column out of the fixed pane (except 'name') and into the scrolling pane, and then close the scrolling pane, I get a HUGE boost in the browser window responsiveness. As soon as I put the waveforms column back into the fixed pane, wham, I get slowed right down. This sucks because being able to audition the file with the waveform is one of the coolest features of Digibase. What could be causing this slowdown? It reminds me of how it would take forever to open a folder in OS9 if you had 'calculate folder sizes' checked.

BTW, it doesn't matter if the waveforms are calculated or not, it's just having the column visible that causes the system to slow down. It also affects the search speed, because it updates the waveform column as the search results are coming in.

Here's a real world example of the drastic difference, searching for 'cobra' yielding 99 results:

With Waveforms Column - 1 minute 25 seconds, extremely sluggish search results window.

Without Waveforms Column - 2 seconds, quick response in search results window.

Does anyone else have this problem?

AB

[stormtrooperpez]

bump

[Paul Greyson]

Interesting. Someone else on the DUC also reported inexplicably lousy DigiBase performance with network drives. I wonder if they were having the same problem.

I'm not sure why having the waveforms displayed should cause a problem. Probably we're pinging the network drive to check for waveforms embedded in the files (I think SoundMiner keeps the overviews in their database instead.)

As a workaround you can set up column layout presets. These should allow you to quickly switch between displaying the waveform column or not. You can toggle between view presets using ctrl 1-5. Also, you don't have to have the waveform column displayed to audition. You can just use the space bar. Not as cool as being able to click on the waveform, but it might get you by until we fix the problem with the overview display.

Thanks for narrowing the problem down. I've been mystified by this.

[stormtrooperpez]

Thanks Paul-

That's exactly what I've been doing - avoiding the waveforms altogether. It certainly proves the usefulness of a customizable browser window!

AB

[Paul Greyson]

So with the workaround are you finding DigiBase to be useful?

[stormtrooperpez]

Quite useful, yes. I wish that there was also some kind of disc ripping utility, however. I've been using iTunes and CDDB to get the track info, but once the long filenames get put into Digibase, they look like this:

07 Computer Monitor W#1F998.aif

Which doesn't tell me a whole lot about what the sound effect is. The file is actually called:

07 Computer Monitor Writing 1

What's with the garbage characters? It makes searches a bit tedious, and renaming all the files by hand in Digibase would takes days and days to do.

But yes, other than that and the waveform issue, I think it's a great tool. I'd really like to see a simple search function directly from the import window. An entire browser seems like overkill sometimes.

AB