Thumbnails mixed up

[jhb50]

My hahasport.groovy displays a flag for the host country for each event title, and my Top100.groovy displays the YouTube thumbnail for each song title.
These are all correct when displaying those feeds on my Samsung C/D profile TV and others have reported no problem with their devices.

Recently I have been testing Mediaplayer under PC Firefox and Android Browser, ServiiGo on my Nextbook Android Tablet, and Serviio via BubbleUPnP on my Nextbook Android Tablet using my BubbleUPnP profile.

The mediabrowser seems OK this am (I think it had the same problem last night) but ServiiGo and Serviio Android are showing the wrong thumbnail for the first 3 or 4 titles and these seem to change randomly and are different on each of these devices. I am testing with the RC. Any thoughts on why the TV is perfect but these devices are randomly different? Only thing I see different in the profiles is keepalive=false for flvplayer.

[zip]

What do you mean Serviio Android? MediaBrowser on android?
Is it just online thumbnails or any local content too?

[jhb50]

What do you mean Serviio Android?

Serviio Android = Serviio via BubbleUPnP on my Nextbook Android Tablet using my BubbleUPnP profile, per viewtopic.php?f=5&t=6558

So both Serviio Android and ServiiGo are messed up but mediabrowser is OK on both PC and Android browser.

Is it just online thumbnails or any local content too?

Had not thought to check locals, but now that I do they seem fine.

[zip]

are they messed up every time you browse to the same folder? are they always the same set of thumbnails, just being assigned to the items in random order? Or are they thumbnails thet belonged to expired items which are no longer there?

[jhb50]

are they messed up every time you browse to the same folder? are they always the same set of thumbnails, just being assigned to the items in random order? Or are they thumbnails thet belonged to expired items which are no longer there?

Well using Serviio Android after a forced refresh on Serviio the Top 100 are now correct but the hahasports are still incorrect and with ServiiGo they are stll both incorrect but the same icons are wrong each time. In the Top100 there are no expired items, but in the hahasport they might be, however the hahasport errors are different in ServiiGo than in Serviio Android.

Its a tough one to diagnose.

[zip]

Maybe ServiiGo caches the thumbnails?

[jhb50]

Maybe ServiiGo caches the thumbnails?

But that does not explain the same failure in using Serviio DLNA with Android BubbleUPnP server.

If I create a cache key based on the content url and its playpath code, and that combination gets used again some days later, the content url will pull the new content, but you have cached the thumbnail for that key so it will display the original thumb rather than the current thumb. That might explain the hahasports problem. I think the same thing happened with Top100 when the titles change position within the list when the next weekly list came out. I need to add a time element to my groovy cached keys, probably the day number the key was created to avoid these reuses.

Consider it solved.

[will]

Maybe ServiiGo caches the thumbnails?

It does, but that can be cleared using Settings -> Clear thumbnail cache. From what we talked about ages ago todo with how the resources ids are generated, it shouldn't be a problem though.

EDIT: Previously ment to say it should not cause a problem. However, you will also need to kill the app if you have already viewed the thumbnails that session as it will not flush them from memory, just disk.

@jhb50 I assume you tried that?

[jhb50]

I was not aware of that ServiiGo setting/action to clear the thumbnail cache. I have now cleared the 262KB of cached thumbnails but they are still mixed up.

However I also went to BubbleUPnp and found that they have a similar cache and when I cleared it, the BubbleUPnP icons are now correct.

So it does appear to be an Android application problem and now only ServiiGo is broken, and the question is why it is still displaying the incorrect icons when the cache shows 0KB.

[will]

See edited post, when you clear the cache it clears the disk cache, anything already in meory will not be evicted. You will need to kill the app before that happens, or go and view some high res photos to push the old ones out of memory. EDIT: Or log out and back in agian.

I'm not familure with the plugin system, but surely if you add a new video it will get a new resource id, rather than resuse one? Edit just read your earlier post so it looks like you will be able to prevent this happening by fixing the plugin.

[jhb50]

Progress:

Looks like there is a bug in ServiiGo. "Clear Cache" does not clear existing icons.

1. In ServiiGo - Settings/Clear thumbnail Cache. Displays 0KB. Return to Folder = still broken and icons display immediately so they are stored somewhere else.
2. Go to Settings/Manage Applications/ServiiGo, Shows Cache = 30KB. Click "Clear Cache". Return to Folder = still broken and icons display immediately so they are stored somewhere else.
3. Go to Settings/Manage Applications/ServiiGo, Shows Data =26KB. Click "Clear Data". Have to reset ServiiGo login & Return to Folder = Icons are empty and rebuild.

Conclusion: Existing Icons are in the applications storage area, and need to be cleared when the refresh menu option is clicked.

PS: I see by your last post that you have same conclusion. And since the refresh in BubbleUPnP gets the correct icons it would seem the plugins are returning the correct icons.

[will]

No it works exactly as it is supposed to, there are two levels of cache, a LRU memory cache sized to device's heap size per app/8 which is ~ 2-6MB. This is used for all types of images, and then the seperate on disk caches for thumbnails, images and online images (one of each for each server). Clearning any of the caches from the app's settings clears the on disk caches.

When you went into android's settings and cleared the cache for SeriiGo, you cleared the http cache. When you cleared the data, you wiped the online metadata database and the preferences, but at the same time killed the app, which cleared the LRU memory cache. Another way to clear the LRU memory cache is to log out and back in again as the cache is dependent on the server, it is cleared on login (I had actually forgot that until I double checked the code ).

The reason why it works this way is because apart from the problem you had, the only reason you would have mixed up thumbnails is because you re-installed serviio which shouldn't happen very often. In that case you can clear the image and thumbnail caches and then logout and back in again and it will be corrected.

[jhb50]

OK, thanks for that clarification.

I should clarify that this plugin changes the content frequently (As often as 3 minutes apart) as sports events start and stop and so the existing icon in memory at a given position needs to be updated for the new event icon. I appreciate that you do not want ServiiGo to automatically refresh the icon with that being sent by Serviio for each item, but the refresh button should be able to do that.(or perhaps a setting to control autorefresh).

I think your note confirms that killing the app does not cause the stored icons to be reset. they remain in the application data area and are still used when ServiiGo is restarted, even if the device is powered off. The only way to clear them currently is to clear all the application data which also clears the logon information and one needs to reinitialize ServiiGo, or to logoff as you explained.

I would have to test more but doesn't the same need exist for news feed where the stories change with each Serviio refresh and the icons are representative of each story?

I probably misunderstand but seems having to logoff to stay in step with dynamically refreshed feeds is not a good solution.

[will]

Your still a little confused by what actually solved your problem/where data is located. The only places images are stored is in memory (the LRU cache which is reset with the app/when you login and out) and at /sdcard/ServiiGo (or similar) which is controlled by ServiiGo's Settings->Clear XXX Cache. There is nothing in the /data/data/com.serviigo directory related to images (that would be bad as on some devices this parition is very small) so doing anything via androids settings thing for serviio go can only kill the app and clear the LRU memory cache.

To clarify: When your plugin refreshes its content, do you re-use cached keys or whatever leading to serviio giving the resource the same id, despite the content being different?

ServiiGo doesn't cache relationships between videos and thumbnails, it just caches the data at the end of the thumbnail url. If you press refresh in the app, and serviio reports that the thumnail for a video is /resources/1283234/thumbnail, and I already have /resources/1283234/thumbnail in the cache then it will not be updated. If you pressed refresh and it now said that the thumbnail had changed and was now /resources/2356485/thumbnail then ServiiGo would fetch the new thumbnail. If that thumbnail was already in cache, ServiiGo would use it.

This all works fine provided whenever a thumbnail is changed, it get a new resource id. So provided this happens, ServiiGo works perfectly well with content that changes never to on every refresh.

[zip]

Might be the problem then. the online IDs are reused and start with 1 at the end (for each online repository), and keep increasing by 1 for each feed item. I think I added the http header for not caching these images on the client side, which is probably why the browser works ok.

Serviio caches them by the thumbnail URL, so it's ok (provided then don't get overwritten), but might serve different image for the same ID after the feed is refreshed.

[will]

So should I not cache thumnails for Online content then. I don't remember seeing a problem with 4oD/ITV Player/YouTube.

[zip]

So should I not cache thumnails for Online content then. I don't remember seeing a problem with 4oD/ITV Player/YouTube.

Probably because they don't update that often, ie the first few are always the same and they just keep adding them?
You might try to do that and for after 1.0 I could look at generating the content IDs for online stuff in a better way.

[jhb50]

Thanks for sticking with me on this, but that does not appear to be the way it is working. I may be totally out to lunch but follow this sequence that occurred this am.

The plugin delivered 5 feed items and I logged into ServiiGo and it displayed the titles and icons correctly. As a reference mediabrowser did the same.

The server plugin subsequently refreshed its contents and items 3 and 4 were removed. The mediabrowser handled this correctly and the former 5th item title and icon became the new 3rd item title and icon. ServiiGo however continued to show the former 3rd item icon with the new 3rd item title, even after clicking refresh. I know that the plugin would have sent a new resource id because the last item and icon is unique and has a unique cachekey ending in "lastitem". Here is the actual data returned by the plugin for the last (3rd)item on the refreshed feed

  Code:

ContentURLContainer [fileType=VIDEO, contentUrl=rtsp://a1709.l1856953708.c18569.
g.lm.akamaistream.net:554/D/1709/18569/v00/reflector:53708, thumbnailUrl=https:/
/sites.google.com/site/serviiorss/nextevent.jpg, live=true, expiresOn=Mon Jul 02
 13:42:00 EDT 2012, expiresImmediately=true, cacheKey=http://www.hahasport.com/c
-11.html_http://lastitem]

I then logged out and logged back in and ServiiGo then displayed the 3rd item icon correctly.

I think this demonstrates that Serviio and the plugin did deliver the correct information for both title and icon when the plugin refreshed, and that ServiiGo recognized the changed items and updated the 3rd title but did not recognize the change in the icon.

Now I know you will say that the resource id for item 3 must not have changed when the plugin refreshed even though the actual item title and icon changed. If that were so why would you have changed the title for the third item in ServiiGo? I think you must have recognized the changed cachekey.

Your description of the logic above is also interesting.

" If you press refresh in the app, and serviio reports that the thumnail for a video is /resources/1283234/thumbnail, and I already have /resources/1283234/thumbnail in the cache then it will not be updated"

If that is the actual logic then it will fail because the new 3rd item thumb is already in the cache having been put there by the previous 5th item.
The logic should also cause the icon to be updated if the cache was cleaned prior to the refresh, but it does not.

Hope this all makes sense.

[will]

OK lets get to the bottom of it. If you aren't already familure with logcat on android, spend a few minutes reading up. I'll edit this post in a min with a build that will show you what it is doing, i'll then post some instructions.

[jhb50]

Might be the problem then. the online IDs are reused and start with 1 at the end (for each online repository), and keep increasing by 1 for each feed item. I think I added the http header for not caching these images on the client side, which is probably why the browser works ok.

Serviio caches them by the thumbnail URL, so it's ok (provided then don't get overwritten), but might serve different image for the same ID after the feed is refreshed.

So how does this explain why the 5th feed item moving up to be the 3rd feed item retains the original icon for the 3rd feed item? If so why does ServiiGo get the correct icons after the plugin refresh when one logs out and restarts? Surely it gets the same info from Serviio as Serviio sent it after the feed refresh. I don't think this explains it. UPnP correctly refreshes the icon after its "Settings/More/Clear album art cache" is cleared. I think its a ServiiGo refresh issue.