CDPedia and itunes artwork

[jschaffe]

Now, is it not possible to only make a call for each album, instead of every track

It's supposed to do that but investigating it a bug was stopping it from optimizing those lookups, thank you for pointing that out. I have fixed CDpedia beta 35, please try it out. It should now make it much further before running into the AppleScript bug.

Hi Conor, I've updated to the latest Snow Leopard, iTunes, and CDPedia, but I still experience dropped artwork. I was wondering if there's any news on this bug.

As a work around, do you think I could import my library in several steps? If so, any guidance on how many albums (or tracks) to import at a time?

Jeff

[Conor]

The next big version will be 10.5+ and we will be able to migrate the AppleScript from a script to a direct bridge to iTunes via the new Scripting Bridge technologies in OS X. Hopefully this will fix the out of memory problem in the AppleScript. In the meantime you can import several separate playlists to break the import. Using a single smart collection that you keep updating should do the trick. You could use the year for example, "songs before 2000", "songs from 2000", "songs from 2001"...

[jschaffe]

The next big version will be 10.5+ and we will be able to migrate the AppleScript from a script to a direct bridge to iTunes via the new Scripting Bridge technologies in OS X. Hopefully this will fix the out of memory problem in the AppleScript.

Great news! Let me know if I can help testing the code against my iTunes library.

In the meantime you can import several separate playlists to break the import. Using a single smart collection that you keep updating should do the trick. You could use the year for example, "songs before 2000", "songs from 2000", "songs from 2001"...

I haven't had time to try this yet, but I have many compilations with songs from various years. What happens if I get some songs from an album in one import and others in a subsequent import?

I'm assuming the procedure goes something like:

a) create smart playlist to select a group of tracks
b) import that playlist
c) make sure all albums have artwork
d) Include collection in library
e) delete the collection
f) edit the smart playlist to select the next group of tracks
g) go to "b"

If I want to select only whole albums, it looks like my only choice is a three-step process:

1) Album artist = blank
2) Album artist = Various Artists
3) Album Artist not blank and not = Various Artists

I can only hope none of those selections is too large. Can you think of any other whole-album selections or parameters that can be applied to the whole album? Or is this something I don't need to worry about?

[Conor]

You bring up a good point that albums need to be kept together. You could use the "compilation is true" smart playlist to process all the albums together that would have songs that fall in different years. For the regular albums with year is in X smart playlist and "compilation is false" would be used to keep those out. In the end it depends very much on the type of albums in iTunes and what would work best for grouping things.

[jschaffe]

You bring up a good point that albums need to be kept together. You could use the "compilation is true" smart playlist to process all the albums together that would have songs that fall in different years. For the regular albums with year is in X smart playlist and "compilation is false" would be used to keep those out. In the end it depends very much on the type of albums in iTunes and what would work best for grouping things.

Well, I did it.

Getting it right involved identifying and fixing quite a few issues in my music library's information.

I started by creating some smart playlists in iTunes:

1. Media kind is music, Album artist = blank, compilation is true
2. Media kind is music, Album artist = blank, compilation is false
3. Media kind is music, Album artist contains various
4. Media kind is music, Album artist is not blank and does not contain Various
5. Media kind is not music

Originally I broke the "compilation is false" list up by year added, but that split a number of albums, so I dispensed with anything date related in dividing up my library.

I found I had to quit both iTunes and CDpedia between each import, or I still missed some album art.

I compared the number of tracks and albums to make sure I was getting everything correct (same numbers as the list above, sorry, the tabs got lost here):

1. iTunes songs: 7,710 iTunes Albums: 503 CDpedia Albums: 503
2. iTunes songs: 12,434 iTunes Albums: 1,250 CDpedia Albums: 1,250
3. iTunes songs: 5,421 iTunes Albums: 2,110 CDpedia Albums: 2,110
4. iTunes songs: 2,688 iTunes Albums: 284 CDpedia Albums: 284
5. iTunes songs: 25 iTunes Albums: -13 (duplicates)

Sub Totals -- iTunes songs: 28,278; iTunes Albums: 4,147; CDpedia Albums: 4,147
Full Library - iTunes songs: 28,278; iTunes Albums: 4,147; CDpedia Albums: 4,147


Other issues that split or merged albums (e.g in iTunes column browser albums column) causing errors or just difficulties in comparing:

• Album titles and Album artists with nearly invisible capitalization or spelling differences ("Live In Paris" vs "Live in Paris", "disc i" vs Disk I")
• Disc <m> of <n> differences
• Track ... of <y> differences
• Sort field differences
• Identical common album titles (anthology, best of, collection, gold, greatest hits, hits, ultimate, very best of, …) (I usually added artist names in parens to differentiate)
• Identical uncommon album titles (The Bridge, one by Billy Joel, another by Melanie Fiona, several others)
• compilation flag set when not needed
• compilation flag not set when needed

I like the fact that this all helped me clean up my database a lot, but it was one bundle of work with many import play lists, delete everything, quit everything and start over. I build a spreadsheet that compared album titles:

1. sort by title in CDpedia
2. select all
3. paste special/text in Excel
4. use the text import wizard to get it right
5. add a column of formulas to displed a "*" if the title is the same as the previous row: =IF(TRIM(LOWER(A638))=TRIM(LOWER(A639)),"*","")

Oh, and by the way...all my CDpedia albums now have artwork.

(whew!)

[Conor]

Thank you for all the details, they will be quite helpful when updating the AppleScript to the new Script bridge technology. With the increased interaction with iTunes I will be able to take more of these special cases into account.

[jschaffe]

Thank you for all the details, they will be quite helpful when updating the AppleScript to the new Script bridge technology. With the increased interaction with iTunes I will be able to take more of these special cases into account.

Unfortunately, I spoke too soon, there are some covers missing

I did a little poking around, and it appears to affect more than one of my smart playlists, but I'm not 100% sure. I'll post more when I know more.

[jschaffe]

Unfortunately, I spoke too soon, there are some covers missing

I did a little poking around, and it appears to affect more than one of my smart playlists, but I'm not 100% sure. I'll post more when I know more.

It was only one of my iTunes smart playlists. It looks like I was successful by dividing my "album artist blank, not compilation" smart play list manually into two regular playlists. I sorted by artist, then selected all tracks whose artist name started with A-G into one list, and artists H-Z into another. As far as I can tell, this worked when I imported the two lists separately.

What's interesting is this is the list that failed:

iTunes songs: 12,434 iTunes Albums: 1,250

while this one succeeded:

iTunes songs: 5,421 iTunes Albums: 2,110

Based on earlier comments about calling iTunes once per album for artwork, I would have expected the second list to fail on import, since it has way more albums. Instead, the list with more tracks but fewer albums failed, so I think there's still some per-track limit being experienced.

In order to succeed, I divided the list into two each with half the tracks, apparently allowing it to succeed:

iTunes songs: 6,080 iTunes Albums: 555
iTunes songs: 6,354 iTunes Albums: 695

By the way, I seem to remember from other discussions that there's no way to create a smart collection of albums without artwork, so I can be sure I got everything. Is that correct?

[Nora]

By the way, I seem to remember from other discussions that there's no way to create a smart collection of albums without artwork, so I can be sure I got everything. Is that correct?

At the moment that's not possible, you're right. We're working on finding a way to include this for the future.

[Conor]

The latest beta should fix any issues for those few having issues with the number of covers that can be imported in a single batch by the AppleScript. It been re-written to use the scripting bridge that does not suffer from the memory bug in the regular AppleScript. For the rest of the users it has the benefit of being faster. The beta can be downloaded from here.

[jschaffe]

Thank you! It's so much easier to simply import everything instead of breaking it up into chunks. As far as I can tell, it's working fine. And even for big collections, it's way faster than before!

The latest beta should fix any issues for those few having issues with the number of covers that can be imported in a single batch by the AppleScript. It been re-written to use the scripting bridge that does not suffer from the memory bug in the regular AppleScript. For the rest of the users it has the benefit of being faster. The beta can be downloaded from here.

[jschaffe]

Oh, one more question. Maybe it's just the beta version or maybe it's been answered elsewhere (I searched though), but the My Ratings values don't seem to match how iTunes sees them.

First, I rate tracks but not albums in iTunes. iTunes calculates it's predicted album rating from those tracks I've rated (I've rated about 75% of my tracks so far, and am listening and rating to shuffled playlists) and displays the value in hollow or empty stars. There are plenty of 4 and 5 star albums in iTunes. I am using the setting that allows me to rate in half-star increments in iTunes.

In version 4.x, CDpedia definitely reported some 4 and 5 star albums. But in CDpedia 5.0.x, the highest-rated albums are just 2.5 stars. In fact, I'm not sure there's even a relationship between what CDpedia shows and what iTunes shows. Blood Sweat & Tears is 5 stars in iTunes, but has no stars at all in the CDpedia beta.

Any ideas what's happening?

[Nora]

There was a bug in the iTunes import regarding the ratings, thanks for pointing that out. It's fixed in the latest CDpedia beta.

[jschaffe]

...the latest CDpedia beta.

I downloaded the beta and installed it and opened it. Then I excluded my last import from my Library and deleted it. Next I started a new import from iTunes and went to lunch. When I returned CDpedia wasn't running. In my console listing I found this:

5/8/12 12:23:13.388 PM CDpedia: Call stack: (
0 Pediabase 0xc0fffd63 -[NSException(NSExceptionExtensions) printStackTrace] + 163
1 Pediabase 0xc0f5a08d -[MyControllerShared(ApplicationNotifications) exceptionHandler:shouldHandleException] + 189
2 ExceptionHandling 0x00212016 -[NSExceptionHandler _handleException] + 1692
3 ExceptionHandling 0x00211972 -[NSExceptionHandler _createAndHandleUncaughtSystemExceptionWithSignal:pc:sp:] + 277
4 ExceptionHandling 0x002122fd NSExceptionHandlerUncaughtSignalHandler + 60
5 libsystem_c.dylib 0x9333c59b _sigtramp + 43
6 ??? 0xffffffff 0x0 + 4294967295
7 CoreFoundation 0x9000c64e __CFStringAppendFormatCore + 12238
8 CoreFoundation 0x9006348c _CFStringCreateWithFormatAndArgumentsAux + 108
9 CoreFoundation 0x900e42fc _CFLogvEx + 124
10 Foundation 0x908a9255 NSLogv + 136
11 Foundation 0x908a91c8 NSLog + 27
12 Pediabase 0xc102e6c3 -[MOEntry setCoverImage:] + 579
13 Pediabase 0xc105c995 +[MyImport lookForiTunesImage:entry:from:] + 357
14 CDpedia 0x0000bdac -[MyController importItunesFromTracks:] + 6028
15 CDpedia 0x0000a575 -[MyController importItunes:] + 2405
16 CoreFoundation 0x9009fde1 -[NSObject performSelector:withObject:] + 65
17 AppKit 0x9733b113 -[NSApplication sendAction:to:from:] + 232
18 AppKit 0x9742ee47 -[NSMenuItem _corePerformAction] + 536
19 AppKit 0x9742eac4 -[NSCarbonMenuImpl performActionWithHighlightingForItemAtIndex:] + 171
20 AppKit 0x9742e14d -[NSMenu _performActionWithHighlightingForItemAtIndex:sendAccessibilityNotification:] + 79
21 AppKit 0x977087e3 -[NSMenu performActionForItemAtIndex:] + 65
22 AppKit 0x97708816 -[NSMenu _internalPerformActionForItemAtIndex:] + 45
23 AppKit 0x9770ca47 -[NSMenuItem _internalPerformActionThroughMenuIfPossible] + 106
24 AppKit 0x97574e58 -[NSCarbonMenuImpl _carbonCommandProcessEvent:handlerCallRef:] + 172
25 AppKit 0x973a3d02 NSSLMMenuEventHandler + 452
26 HIToolbox 0x9b8c962c _Z22_InvokeEventHandlerUPPP25OpaqueEventHandlerCallRefP14OpaqueEventRefPvPFlS0_S2_S3_E + 36
27 HIToolbox 0x9b744f4f _ZL23DispatchEventToHandlersP14EventTargetRecP14OpaqueEventRefP14HandlerCallRec + 1602
28 HIToolbox 0x9b7443cc _ZL30SendEventToEventTargetInternalP14OpaqueEventRefP20OpaqueEventTargetRefP14HandlerCallRec + 482
29 HIToolbox 0x9b7590d5 SendEventToEventTarget + 76
30 HIToolbox 0x9b8c9af0 _ZL18SendHICommandEventmPK9HICommandmmhPKvP20OpaqueEventTargetRefS5_PP14OpaqueEventRef + 482
31 HIToolbox 0x9b8c9b5a SendMenuCommandWithContextAndModifiers + 70
32 HIToolbox 0x9b934631 SendMenuItemSelectedEvent + 275
33 HIToolbox 0x9b795881 _ZL19FinishMenuSelectionP13SelectionDataP10MenuResultS2_ + 129
34 HIToolbox 0x9b925530 _ZL14MenuSelectCoreP8MenuData5PointdmPP13OpaqueMenuRefPt + 608
35 HIToolbox 0x9b78d52e _HandleMenuSelection2 + 636
36 HIToolbox 0x9b78d2ad _HandleMenuSelection + 53
37 AppKit 0x9729fd4a _NSHandleCarbonMenuEvent + 302
38 AppKit 0x9723008a _DPSNextEvent + 2196
39 AppKit 0x9722f306 -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 113
40 AppKit 0x9722b675 -[NSApplication run] + 911
41 AppKit 0x974bf261 NSApplicationMain + 1054
42 CDpedia 0x0000215b main + 43
43 CDpedia 0x000020e5 start + 53
)
5/8/12 12:23:13.388 PM CDpedia: An uncaught exception was raised
5/8/12 12:23:13.388 PM CDpedia: Uncaught system exception: signal 11
5/8/12 12:23:13.390 PM CDpedia: (
0 CoreFoundation 0x9014ab67 __raiseError + 231
1 libobjc.A.dylib 0x9657b149 objc_exception_throw + 155
2 ExceptionHandling 0x00211f08 -[NSExceptionHandler _handleException] + 1422
3 ExceptionHandling 0x00211972 -[NSExceptionHandler _createAndHandleUncaughtSystemExceptionWithSignal:pc:sp:] + 277
4 ExceptionHandling 0x002122fd NSExceptionHandlerUncaughtSignalHandler + 60
5 libsystem_c.dylib 0x9333c59b _sigtramp + 43
6 ??? 0xffffffff 0x0 + 4294967295
7 CoreFoundation 0x9000c64e __CFStringAppendFormatCore + 12238
8 CoreFoundation 0x9006348c _CFStringCreateWithFormatAndArgumentsAux + 108
9 CoreFoundation 0x900e42fc _CFLogvEx + 124
10 Foundation 0x908a9255 NSLogv + 136
11 Foundation 0x908a91c8 NSLog + 27
12 Pediabase 0xc102e6c3 -[MOEntry setCoverImage:] + 579
13 Pediabase 0xc105c995 +[MyImport lookForiTunesImage:entry:from:] + 357
14 CDpedia 0x0000bdac -[MyController importItunesFromTracks:] + 6028
15 CDpedia 0x0000a575 -[MyController importItunes:] + 2405
16 CoreFoundation 0x9009fde1 -[NSObject performSelector:withObject:] + 65
17 AppKit 0x9733b113 -[NSApplication sendAction:to:from:] + 232
18 AppKit 0x9742ee47 -[NSMenuItem _corePerformAction] + 536
19 AppKit 0x9742eac4 -[NSCarbonMenuImpl performActionWithHighlightingForItemAtIndex:] + 171
20 AppKit 0x9742e14d -[NSMenu _performActionWithHighlightingForItemAtIndex:sendAccessibilityNotification:] + 79
21 AppKit 0x977087e3 -[NSMenu performActionForItemAtIndex:] + 65
22 AppKit 0x97708816 -[NSMenu _internalPerformActionForItemAtIndex:] + 45
23 AppKit 0x9770ca47 -[NSMenuItem _internalPerformActionThroughMenuIfPossible] + 106
24 AppKit 0x97574e58 -[NSCarbonMenuImpl _carbonCommandProcessEvent:handlerCallRef:] + 172
25 AppKit 0x973a3d02 NSSLMMenuEventHandler + 452
26 HIToolbox 0x9b8c962c _Z22_InvokeEventHandlerUPPP25OpaqueEventHandlerCallRefP14OpaqueEventRefPvPFlS0_S2_S3_E + 36
27 HIToolbox 0x9b744f4f _ZL23DispatchEventToHandlersP14EventTargetRecP14OpaqueEventRefP14HandlerCallRec + 1602
28 HIToolbox 0x9b7443cc _ZL30SendEventToEventTargetInternalP14OpaqueEventRefP20OpaqueEventTargetRefP14HandlerCallRec + 482
29 HIToolbox 0x9b7590d5 SendEventToEventTarget + 76
30 HIToolbox 0x9b8c9af0 _ZL18SendHICommandEventmPK9HICommandmmhPKvP20OpaqueEventTargetRefS5_PP14OpaqueEventRef + 482
31 HIToolbox 0x9b8c9b5a SendMenuCommandWithContextAndModifiers + 70
32 HIToolbox 0x9b934631 SendMenuItemSelectedEvent + 275
33 HIToolbox 0x9b795881 _ZL19FinishMenuSelectionP13SelectionDataP10MenuResultS2_ + 129
34 HIToolbox 0x9b925530 _ZL14MenuSelectCoreP8MenuData5PointdmPP13OpaqueMenuRefPt + 608
35 HIToolbox 0x9b78d52e _HandleMenuSelection2 + 636
36 HIToolbox 0x9b78d2ad _HandleMenuSelection + 53
37 AppKit 0x9729fd4a _NSHandleCarbonMenuEvent + 302
38 AppKit 0x9723008a _DPSNextEvent + 2196
39 AppKit 0x9722f306 -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 113
40 AppKit 0x9722b675 -[NSApplication run] + 911
41 AppKit 0x974bf261 NSApplicationMain + 1054
42 CDpedia 0x0000215b main + 43
43 CDpedia 0x000020e5 start + 53
)
5/8/12 12:23:13.391 PM CDpedia: Call stack: (
0 Pediabase 0xc0fffd63 -[NSException(NSExceptionExtensions) printStackTrace] + 163
1 Pediabase 0xc0f5a08d -[MyControllerShared(ApplicationNotifications) exceptionHandler:shouldHandleException] + 189
2 ExceptionHandling 0x00212016 -[NSExceptionHandler _handleException] + 1692
3 ExceptionHandling 0x002122a7 NSExceptionHandlerUncaughtExceptionHandler + 331
4 HIToolbox 0x9b9f7ed3 HIReportUnhandledException + 42
5 HIToolbox 0x9b744f15 _ZL23DispatchEventToHandlersP14EventTargetRecP14OpaqueEventRefP14HandlerCallRec + 1544
6 HIToolbox 0x9b7443cc _ZL30SendEventToEventTargetInternalP14OpaqueEventRefP20OpaqueEventTargetRefP14HandlerCallRec + 482
7 HIToolbox 0x9b7590d5 SendEventToEventTarget + 76
8 HIToolbox 0x9b8c9af0 _ZL18SendHICommandEventmPK9HICommandmmhPKvP20OpaqueEventTargetRefS5_PP14OpaqueEventRef + 482
9 HIToolbox 0x9b8c9b5a SendMenuCommandWithContextAndModifiers + 70
10 HIToolbox 0x9b934631 SendMenuItemSelectedEvent + 275
11 HIToolbox 0x9b795881 _ZL19FinishMenuSelectionP13SelectionDataP10MenuResultS2_ + 129
12 HIToolbox 0x9b925530 _ZL14MenuSelectCoreP8MenuData5PointdmPP13OpaqueMenuRefPt + 608
13 HIToolbox 0x9b78d52e _HandleMenuSelection2 + 636
14 HIToolbox 0x9b78d2ad _HandleMenuSelection + 53
15 AppKit 0x9729fd4a _NSHandleCarbonMenuEvent + 302
16 AppKit 0x9723008a _DPSNextEvent + 2196
17 AppKit 0x9722f306 -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 113
18 AppKit 0x9722b675 -[NSApplication run] + 911
19 AppKit 0x974bf261 NSApplicationMain + 1054
20 CDpedia 0x0000215b main + 43
21 CDpedia 0x000020e5 start + 53
)
5/8/12 12:23:13.393 PM CDpedia: Call stack: (
0 Pediabase 0xc0fffd63 -[NSException(NSExceptionExtensions) printStackTrace] + 163
1 Pediabase 0xc0f5a08d -[MyControllerShared(ApplicationNotifications) exceptionHandler:shouldHandleException] + 189
2 ExceptionHandling 0x00212016 -[NSExceptionHandler _handleException] + 1692
3 ExceptionHandling 0x00211972 -[NSExceptionHandler _createAndHandleUncaughtSystemExceptionWithSignal:pc:sp:] + 277
4 ExceptionHandling 0x002122fd NSExceptionHandlerUncaughtSignalHandler + 60
5 libsystem_c.dylib 0x9333c59b _sigtramp + 43
6 ??? 0xffffffff 0x0 + 4294967295
)
5/8/12 12:23:13.393 PM CDpedia: An uncaught exception was raised
5/8/12 12:23:13.393 PM CDpedia: Uncaught system exception: signal 10
5/8/12 12:23:13.393 PM CDpedia: (
0 CoreFoundation 0x9014ab67 __raiseError + 231
1 libobjc.A.dylib 0x9657b149 objc_exception_throw + 155
2 ExceptionHandling 0x00211f08 -[NSExceptionHandler _handleException] + 1422
3 ExceptionHandling 0x00211972 -[NSExceptionHandler _createAndHandleUncaughtSystemExceptionWithSignal:pc:sp:] + 277
4 ExceptionHandling 0x002122fd NSExceptionHandlerUncaughtSignalHandler + 60
5 libsystem_c.dylib 0x9333c59b _sigtramp + 43
6 ??? 0xffffffff 0x0 + 4294967295
)
5/8/12 12:23:13.393 PM CDpedia: Call stack: (
0 Pediabase 0xc0fffd63 -[NSException(NSExceptionExtensions) printStackTrace] + 163
1 Pediabase 0xc0f5a08d -[MyControllerShared(ApplicationNotifications) exceptionHandler:shouldHandleException] + 189
2 ExceptionHandling 0x00212016 -[NSExceptionHandler _handleException] + 1692
3 ExceptionHandling 0x002122a7 NSExceptionHandlerUncaughtExceptionHandler + 331
4 CoreFoundation 0x9014ac19 __raiseError + 409
5 libobjc.A.dylib 0x9657b149 objc_exception_throw + 155
6 ExceptionHandling 0x00211f08 -[NSExceptionHandler _handleException] + 1422
7 ExceptionHandling 0x00211972 -[NSExceptionHandler _createAndHandleUncaughtSystemExceptionWithSignal:pc:sp:] + 277
8 ExceptionHandling 0x002122fd NSExceptionHandlerUncaughtSignalHandler + 60
9 libsystem_c.dylib 0x9333c59b _sigtramp + 43
10 ??? 0xffffffff 0x0 + 4294967295
)

[jschaffe]

I wonder...I started playing music from iTunes around that time...any possibility it triggered the crash?