Thumbnails mixed up

[jhb50]

Progress:

Looks like there is a bug in ServiiGo. "Clear Cache" does not clear existing icons.

1. In ServiiGo - Settings/Clear thumbnail Cache. Displays 0KB. Return to Folder = still broken and icons display immediately so they are stored somewhere else.
2. Go to Settings/Manage Applications/ServiiGo, Shows Cache = 30KB. Click "Clear Cache". Return to Folder = still broken and icons display immediately so they are stored somewhere else.
3. Go to Settings/Manage Applications/ServiiGo, Shows Data =26KB. Click "Clear Data". Have to reset ServiiGo login & Return to Folder = Icons are empty and rebuild.

Conclusion: Existing Icons are in the applications storage area, and need to be cleared when the refresh menu option is clicked.

PS: I see by your last post that you have same conclusion. And since the refresh in BubbleUPnP gets the correct icons it would seem the plugins are returning the correct icons.

[will]

No it works exactly as it is supposed to, there are two levels of cache, a LRU memory cache sized to device's heap size per app/8 which is ~ 2-6MB. This is used for all types of images, and then the seperate on disk caches for thumbnails, images and online images (one of each for each server). Clearning any of the caches from the app's settings clears the on disk caches.

When you went into android's settings and cleared the cache for SeriiGo, you cleared the http cache. When you cleared the data, you wiped the online metadata database and the preferences, but at the same time killed the app, which cleared the LRU memory cache. Another way to clear the LRU memory cache is to log out and back in again as the cache is dependent on the server, it is cleared on login (I had actually forgot that until I double checked the code ).

The reason why it works this way is because apart from the problem you had, the only reason you would have mixed up thumbnails is because you re-installed serviio which shouldn't happen very often. In that case you can clear the image and thumbnail caches and then logout and back in again and it will be corrected.

[jhb50]

OK, thanks for that clarification.

I should clarify that this plugin changes the content frequently (As often as 3 minutes apart) as sports events start and stop and so the existing icon in memory at a given position needs to be updated for the new event icon. I appreciate that you do not want ServiiGo to automatically refresh the icon with that being sent by Serviio for each item, but the refresh button should be able to do that.(or perhaps a setting to control autorefresh).

I think your note confirms that killing the app does not cause the stored icons to be reset. they remain in the application data area and are still used when ServiiGo is restarted, even if the device is powered off. The only way to clear them currently is to clear all the application data which also clears the logon information and one needs to reinitialize ServiiGo, or to logoff as you explained.

I would have to test more but doesn't the same need exist for news feed where the stories change with each Serviio refresh and the icons are representative of each story?

I probably misunderstand but seems having to logoff to stay in step with dynamically refreshed feeds is not a good solution.

[will]

Your still a little confused by what actually solved your problem/where data is located. The only places images are stored is in memory (the LRU cache which is reset with the app/when you login and out) and at /sdcard/ServiiGo (or similar) which is controlled by ServiiGo's Settings->Clear XXX Cache. There is nothing in the /data/data/com.serviigo directory related to images (that would be bad as on some devices this parition is very small) so doing anything via androids settings thing for serviio go can only kill the app and clear the LRU memory cache.

To clarify: When your plugin refreshes its content, do you re-use cached keys or whatever leading to serviio giving the resource the same id, despite the content being different?

ServiiGo doesn't cache relationships between videos and thumbnails, it just caches the data at the end of the thumbnail url. If you press refresh in the app, and serviio reports that the thumnail for a video is /resources/1283234/thumbnail, and I already have /resources/1283234/thumbnail in the cache then it will not be updated. If you pressed refresh and it now said that the thumbnail had changed and was now /resources/2356485/thumbnail then ServiiGo would fetch the new thumbnail. If that thumbnail was already in cache, ServiiGo would use it.

This all works fine provided whenever a thumbnail is changed, it get a new resource id. So provided this happens, ServiiGo works perfectly well with content that changes never to on every refresh.

[zip]

Might be the problem then. the online IDs are reused and start with 1 at the end (for each online repository), and keep increasing by 1 for each feed item. I think I added the http header for not caching these images on the client side, which is probably why the browser works ok.

Serviio caches them by the thumbnail URL, so it's ok (provided then don't get overwritten), but might serve different image for the same ID after the feed is refreshed.

[will]

So should I not cache thumnails for Online content then. I don't remember seeing a problem with 4oD/ITV Player/YouTube.

[zip]

So should I not cache thumnails for Online content then. I don't remember seeing a problem with 4oD/ITV Player/YouTube.

Probably because they don't update that often, ie the first few are always the same and they just keep adding them?
You might try to do that and for after 1.0 I could look at generating the content IDs for online stuff in a better way.

[jhb50]

Thanks for sticking with me on this, but that does not appear to be the way it is working. I may be totally out to lunch but follow this sequence that occurred this am.

The plugin delivered 5 feed items and I logged into ServiiGo and it displayed the titles and icons correctly. As a reference mediabrowser did the same.

The server plugin subsequently refreshed its contents and items 3 and 4 were removed. The mediabrowser handled this correctly and the former 5th item title and icon became the new 3rd item title and icon. ServiiGo however continued to show the former 3rd item icon with the new 3rd item title, even after clicking refresh. I know that the plugin would have sent a new resource id because the last item and icon is unique and has a unique cachekey ending in "lastitem". Here is the actual data returned by the plugin for the last (3rd)item on the refreshed feed

  Code:

ContentURLContainer [fileType=VIDEO, contentUrl=rtsp://a1709.l1856953708.c18569.
g.lm.akamaistream.net:554/D/1709/18569/v00/reflector:53708, thumbnailUrl=https:/
/sites.google.com/site/serviiorss/nextevent.jpg, live=true, expiresOn=Mon Jul 02
 13:42:00 EDT 2012, expiresImmediately=true, cacheKey=http://www.hahasport.com/c
-11.html_http://lastitem]

I then logged out and logged back in and ServiiGo then displayed the 3rd item icon correctly.

I think this demonstrates that Serviio and the plugin did deliver the correct information for both title and icon when the plugin refreshed, and that ServiiGo recognized the changed items and updated the 3rd title but did not recognize the change in the icon.

Now I know you will say that the resource id for item 3 must not have changed when the plugin refreshed even though the actual item title and icon changed. If that were so why would you have changed the title for the third item in ServiiGo? I think you must have recognized the changed cachekey.

Your description of the logic above is also interesting.

" If you press refresh in the app, and serviio reports that the thumnail for a video is /resources/1283234/thumbnail, and I already have /resources/1283234/thumbnail in the cache then it will not be updated"

If that is the actual logic then it will fail because the new 3rd item thumb is already in the cache having been put there by the previous 5th item.
The logic should also cause the icon to be updated if the cache was cleaned prior to the refresh, but it does not.

Hope this all makes sense.

[will]

OK lets get to the bottom of it. If you aren't already familure with logcat on android, spend a few minutes reading up. I'll edit this post in a min with a build that will show you what it is doing, i'll then post some instructions.

[jhb50]

Might be the problem then. the online IDs are reused and start with 1 at the end (for each online repository), and keep increasing by 1 for each feed item. I think I added the http header for not caching these images on the client side, which is probably why the browser works ok.

Serviio caches them by the thumbnail URL, so it's ok (provided then don't get overwritten), but might serve different image for the same ID after the feed is refreshed.

So how does this explain why the 5th feed item moving up to be the 3rd feed item retains the original icon for the 3rd feed item? If so why does ServiiGo get the correct icons after the plugin refresh when one logs out and restarts? Surely it gets the same info from Serviio as Serviio sent it after the feed refresh. I don't think this explains it. UPnP correctly refreshes the icon after its "Settings/More/Clear album art cache" is cleared. I think its a ServiiGo refresh issue.

[jhb50]

OK lets get to the bottom of it. If you aren't already familure with logcat on android, spend a few minutes reading up. I'll edit this post in a min with a build that will show you what it is doing, i'll then post some instructions.

Happy to help! What's another day LOL!

[will]

There may be a bug in ServiiGo, but I'm not sure how it could cause this problem (Instead the problem would be reused resource ids for the thumbnails).

Download: https://bitbucket.org/willlunniss/servi ... -DEBUG.apk

Do you have adb setup on your PC?

If so open up two consoles. In the first type (otherwise set it up or try one of these apps http://forum.xda-developers.com/showthr ... ?t=1726238)

  Code:

adb logcat -s "ServiiGo" > ServiiGo.log.txt


to keep a record and in the second type

  Code:

adb logcat -s "ServiiGo"


If an image isn't in cache you will get this chain of events (note it is a first in last out queue so they will go in one order, and come out in reverse (except for the first entry):

  Code:

D/ServiiGo(23069): [ImageManager:displayImage] Queuing image : http://192.168.15.100:23424/cds/resource/1000000000020004/COVER_IMAGE,serviigo?authToken=80b1b5468ee64a31bf460b145affe36e

D/ServiiGo(23069): [ImageManager:getBitmap] Url : http://192.168.15.100:23424/cds/resource/1000000000020004/COVER_IMAGE,serviigo?authToken=80b1b5468ee64a31bf460b145affe36e

D/ServiiGo(23069): [ImageManager:writeFile] File : file:/mnt/sdcard/ServiiGo/ThumbnailCache/192.168.15.100-23424/-1133382581

D/ServiiGo(23069): [ImageManager:displayImage] Url : http://192.168.15.100:23424/cds/resource/1000000000020004/COVER_IMAGE,serviigo?authToken=80b1b5468ee64a31bf460b145affe36e


If an image is in the disk cache, but not the memory one you will get:

  Code:

D/ServiiGo(23069): [ImageManager:displayImage] Queuing image : http://192.168.15.100:23424/cds/resource/1000000000020004/COVER_IMAGE,serviigo?authToken=ecb6c803696346629baaea94b000bc95

D/ServiiGo(23069): [ImageManager:getBitmap] From SD Card : /mnt/sdcard/ServiiGo/ThumbnailCache/192.168.15.100-23424/-1133382581

D/ServiiGo(23069): [ImageManager:displayImage] Url : http://192.168.15.100:23424/cds/resource/1000000000020004/COVER_IMAGE,serviigo?authToken=ecb6c803696346629baaea94b000bc95


and finally, if the image is in the memory cache you will get:

  Code:

D/ServiiGo(23069): [ImageManager:displayImage] From cache : http://192.168.15.100:23424/cds/resource/1000000000020004/COVER_IMAGE,serviigo?authToken=ecb6c803696346629baaea94b000bc95


So can you go through those steps you did before and at each stage check manually what the image is, to see if the image changes while still keeping the same url.

Additionally, check the data comming from Serviio by looking for a line like this and appending http://server:23424/cds/ to it

  Code:

D/ServiiGo(23069): [Application:browse] Url : browse/serviigo/V_OF%5EFOL_FD2/BrowseDirectChildren/all/0/0?authToken=ecb6c803696346629baaea94b000bc95


If you get stuck, post the whole log and I will look through it. Provided you access your server locally, there is nothing in there of any use so it is safe to post.

So some points to look for, in my example, I would keep http://192.168.15.100:23424/cds/browse/ ... 94b000bc95 open in my browser, and refresh to watch the content change.
If the content title changed, but the thumbnail under thumbnailUrl didn't change (which is what I / zip expect), then serviigo will display the wrong thumbnail. This can be fixed by not caching thumbnails for online content.

[jhb50]

Might be the problem then. the online IDs are reused and start with 1 at the end (for each online repository), and keep increasing by 1 for each feed item. I think I added the http header for not caching these images on the client side, which is probably why the browser works ok.

Serviio caches them by the thumbnail URL, so it's ok (provided then don't get overwritten), but might serve different image for the same ID after the feed is refreshed.

So how does this explain why the 5th feed item moving up to be the 3rd feed item retains the original icon for the 3rd feed item? If so why does ServiiGo get the correct icons after the plugin refresh when one logs out and restarts? Surely it gets the same info from Serviio as Serviio sent it after the feed refresh. I don't think this explains it. UPnP correctly refreshes the icon after its "Settings/More/Clear album art cache" is cleared. I think its a ServiiGo refresh issue.

Thinking this over, if he sends back the same id(which btw is not the new cachekey)'for the new 3rd item then you will assume(rightfully so)that the icon has not changed. You will only get the correct icon when you need to rebuild the cache. Why dont you update the icons as well when I click Refresh in ServiiGo? It would solve all these problems.

[will]

Thinking this over, if he sends back the same id(which btw is not the new cachekey)'for the new 3rd item then you will assume(rightfully so)that the icon has not changed. You will only get the correct icon when you need to rebuild the cache. Why dont you update the icons as well when I click Refresh in ServiiGo? It would solve all these problems.

Do you mean when you press refresh, go through all the current thumbnails belonging to the current content and delete them from the memory cache and delete them from the disk cache? I guess I could do that for Online folders only. Although that would add a noticable delay on some devices when there are a large number of items, but would be better than not caching them for users which use online content that doesn't have this problem. Will take me a little while to implement, I'll try and send you a test build tomorrow.

However, if this is the problem, then it will keep coming back in apps like BubleUPnP etc...

[will]

OK here is a quick fix. If this works I will improve it future for 1.0 by not saving the images to the sd card in the first place, for now when refreshing an online folder, I go and clear out any thumbnails from memory and disk. So logging out then loading an online folder will shown the wrong thumnails, until you refresh. Once I implement the proper workaround, this won't be needed.

https://bitbucket.org/willlunniss/servi ... -DEBUG.apk

[jhb50]

Re your last 2 notes, once you clear the memory and cache, I think all that will happen is that the current few displayed icons in each folder will show the film strip and will need to be reloaded from Serviio. That is no more delay than when users go to the next few icons for the first time.
You are correct about Bubble. It shows the wrong icons now but has a clear setting that I posted above that clears the memory and causes the correct icons to be displayed.

Once zip fixes it to send a new id when the item changes, I think everything will work ok without you doing anything and you I think you should continue to cache, but I think having the easy to access refresh of cache and memory is a worthwhile feature anyway that isolates ServiiGo from anyones screwups. When something does not work its real nice to be able to start clean.

On the other issue of long start times for online feeds, I think it might be MX player. A quick test using VPlayer started quickly. I'll test more.

[zip]

added the ticket https://bitbucket.org/xnejp03/serviio/i ... ms-ids-are

[jhb50]

OK here is a quick fix.

Worked like a charm!

Logged on, opened feed, 2 items now containing the icons from the previous items 1 an 2. Pressed refresh, both items switched to film icon then quickly populated with the correct icons.

I will work on the stream start delay now and post over in ServiiGo thread.

Consider this SOLVED!

[will]

Thats good.

Here is the revised version I talked about which won't store the images on the SD card in the first place, plus debug logging is turned off. https://bitbucket.org/willlunniss/servi ... .0-PRE.apk

[jhb50]

Ooops...that one does not work. Displays previous bad icons and refresh does not correct them. Reverted to DEBUG version and refresh corrects them correctly.