FAQ  •  Register  •  Login

DB size growing

<<

josch.hh

Serviio lover

Posts: 66

Joined: Sun Dec 20, 2015 10:01 pm

Post Mon Dec 21, 2015 12:55 am

DB size growing

It seems that the database is not well cleaned after removing a complete folder from the library.
The DB was about 597MB size including the folder i am talking about.
After removing it the DB got somerthing about 590MB.
After adding it again the size is now 625MB an growing....

Seems not ok for me.

Is there an option to clean up the DB files?
<<

jhb50

DLNA master

Posts: 2843

Joined: Thu Jun 30, 2011 9:32 pm

Post Wed Dec 23, 2015 3:00 am

Re: DB size growing

I believe the only way is to delete it and let it rebuild.
<<

zip

User avatar

Serviio developer / Site Admin

Posts: 17215

Joined: Sat Oct 24, 2009 12:24 pm

Location: London, UK

Post Wed Dec 23, 2015 1:28 pm

Re: DB size growing

Serviio uses Apache Derby DB. I don't know the inner workings but it's possible it doesn't shrink the data blobs in order to keep consistency and performance.
<<

skiaccianos

User avatar

Streaming enthusiast

Posts: 28

Joined: Thu Nov 26, 2015 2:06 pm

Location: Terni, Umbria, Italia

Post Wed Dec 23, 2015 6:57 pm

Re: DB size growing

Btw, zip: I've tested the very latest derby.jar with serviio. It works fine and has same weight.

Inviato dal mio Moto G utilizzando Tapatalk
Running Serviio - for several years now - on Debian and Win boxes.
R850R biker, sysadmin & DBA, FOSS and Linux enthusiast (desktop_is_home_dir)
<<

josch.hh

Serviio lover

Posts: 66

Joined: Sun Dec 20, 2015 10:01 pm

Post Wed Dec 23, 2015 8:08 pm

Re: DB size growing

skiaccianos wrote:Btw, zip: I've tested the very latest derby.jar with serviio. It works fine and has same weight.

Inviato dal mio Moto G utilizzando Tapatalk


And is the behaviour concerning the database cleanup for remove folders from serviio better,.... or what is the improvement?
<<

josch.hh

Serviio lover

Posts: 66

Joined: Sun Dec 20, 2015 10:01 pm

Post Wed Dec 23, 2015 8:15 pm

Re: DB size growing

zip wrote:Serviio uses Apache Derby DB. I don't know the inner workings but it's possible it doesn't shrink the data blobs in order to keep consistency and performance.


Mmh, so what is the consequence? If one is regularly updating or modifying his media files a recreation of the derby db is needed to avoid that the database gets bigger than the source media files it points to?
<<

DenyAll

DLNA master

Posts: 2257

Joined: Fri Mar 08, 2013 11:16 pm

Location: Adelaide, Australia

Post Thu Dec 24, 2015 1:02 am

Re: DB size growing

Bearing in mind I know nothing about derby:
This post http://apache-database.10148.n7.nabble. ... 04961.html (assuming that the response in that forum is accurate) begs the question - does Serviio periodically compress the database programatically? Is it practical to do so (time taken to compress, etc)? If not, perhaps by a third party tool?
DenyAll
Panasonic Viera FX800A | Panasonic Viera CS610A | Sony PS4 | Sony PS3 | Panasonic DMP-BD79 | Yamaha RX-V500D | iPad | Windows 10 | Serviio 1.10.1 Pro
WinHelper | MediaInfo

Beta Tester, Moderator
Please do not PM me for support as any solution cannot be shared with others.
<<

josch.hh

Serviio lover

Posts: 66

Joined: Sun Dec 20, 2015 10:01 pm

Post Thu Dec 24, 2015 8:49 am

Re: DB size growing

Hi again, dont get me wrong, i really like Serviio, but as the DB is a major part i guess you should be sure about its behaviour in combination with the functionallity of your software.
Dont you think it is worth to take a deeper look from your side... i think the source cocde of serviio is not published so you are the only one to check to collaboration.

What i found was the following:

https://db.apache.org/derby/docs/10.1/adminguide/cadminspace21579.html
http://www-01.ibm.com/support/docview.wss?uid=swg21577292

The last link even with some java exmaples.
<<

skiaccianos

User avatar

Streaming enthusiast

Posts: 28

Joined: Thu Nov 26, 2015 2:06 pm

Location: Terni, Umbria, Italia

Post Thu Dec 24, 2015 1:01 pm

Re: DB size growing

josch.hh wrote:
skiaccianos wrote:Btw, zip: I've tested the very latest derby.jar with serviio. It works fine and has same weight.

Inviato dal mio Moto G utilizzando Tapatalk


And is the behaviour concerning the database cleanup for remove folders from serviio better,.... or what is the improvement?


"... Derby 10.12.1.1 contains many bug, security, and documentation fixes."

If there's a problem, the first thing to do is to get the latest release. :) About me, I've never had a so huge an half gb db size: my .serviio/library folder has always been around 10 mb or less.
Running Serviio - for several years now - on Debian and Win boxes.
R850R biker, sysadmin & DBA, FOSS and Linux enthusiast (desktop_is_home_dir)
<<

josch.hh

Serviio lover

Posts: 66

Joined: Sun Dec 20, 2015 10:01 pm

Post Mon Dec 28, 2015 8:44 am

Re: DB size growing

skiaccianos wrote:
josch.hh wrote:
skiaccianos wrote:Btw, zip: I've tested the very latest derby.jar with serviio. It works fine and has same weight.

Inviato dal mio Moto G utilizzando Tapatalk


And is the behaviour concerning the database cleanup for remove folders from serviio better,.... or what is the improvement?


"... Derby 10.12.1.1 contains many bug, security, and documentation fixes."

If there's a problem, the first thing to do is to get the latest release. :) About me, I've never had a so huge an half gb db size: my .serviio/library folder has always been around 10 mb or less.


Concerning my DB size: It contains about 500 Movies and maybe 8000 mp3 and about 2000 pictures
... i dont think is this really much
Minidlna consumed only 20% of the db files size and art cache compared with serviio.
<<

zip

User avatar

Serviio developer / Site Admin

Posts: 17215

Joined: Sat Oct 24, 2009 12:24 pm

Location: London, UK

Post Fri Jan 01, 2016 2:58 pm

Re: DB size growing

minidlna also doesn't store all the descriptive metadata, like summary, actors, 2 sizes of thumbnails, etc.
<<

josch.hh

Serviio lover

Posts: 66

Joined: Sun Dec 20, 2015 10:01 pm

Post Fri Jan 01, 2016 3:43 pm

Re: DB size growing

ok, but i dont think that this results in an x times bigger DB
<<

zip

User avatar

Serviio developer / Site Admin

Posts: 17215

Joined: Sat Oct 24, 2009 12:24 pm

Location: London, UK

Post Fri Jan 01, 2016 8:29 pm

Re: DB size growing

It could. And obviously they both use different DB engines, so you cannot really compare them anyway.
<<

jhb50

DLNA master

Posts: 2843

Joined: Thu Jun 30, 2011 9:32 pm

Post Sat Jan 02, 2016 1:54 am

Re: DB size growing

What did we conclude here about cleaning old stuff out of the db?
My concern is my on-line streams for which the stream urls change regularly, as opposed to files which are static.
Am I correct that "the only way is to delete it and let it rebuild."??
<<

zip

User avatar

Serviio developer / Site Admin

Posts: 17215

Joined: Sat Oct 24, 2009 12:24 pm

Location: London, UK

Post Sat Jan 02, 2016 7:21 pm

Re: DB size growing

Online streams are not stored into the DB (Derby), but use a caching mechanism (Apache JCS). No idea how the space in the persistent cache is reclaimed... but i guess you already have better way to test it than me (I hardly use any online sources personally).
<<

jhb50

DLNA master

Posts: 2843

Joined: Thu Jun 30, 2011 9:32 pm

Post Sat Jan 02, 2016 8:19 pm

Re: DB size growing

zip wrote:Online streams are not stored into the DB (Derby), but use a caching mechanism (Apache JCS). No idea how the space in the persistent cache is reclaimed... but i guess you already have better way to test it than me (I hardly use any online sources personally).

So online metadata is stored in C:\Program Files\Serviio\library\cache\online_technical_metadata.data (Mine is only 5MB) and the thumbs in and the thumbnails in C:\Program Files\Serviio\library\cache\thumbnails.data (Mine is 99MB). Files metadata is in C:\Program Files\Serviio\library\db (Mine is 282MB)
<<

josch.hh

Serviio lover

Posts: 66

Joined: Sun Dec 20, 2015 10:01 pm

Post Sat Jan 02, 2016 8:43 pm

Re: DB size growing

jhb50 wrote:
zip wrote:Online streams are not stored into the DB (Derby), but use a caching mechanism (Apache JCS). No idea how the space in the persistent cache is reclaimed... but i guess you already have better way to test it than me (I hardly use any online sources personally).

So online metadata is stored in C:\Program Files\Serviio\library\cache\online_technical_metadata.data (Mine is only 5MB) and the thumbs in and the thumbnails in C:\Program Files\Serviio\library\cache\thumbnails.data (Mine is 99MB). Files metadata is in C:\Program Files\Serviio\library\db (Mine is 282MB)


How many files of which kind are in your db?
<<

jhb50

DLNA master

Posts: 2843

Joined: Thu Jun 30, 2011 9:32 pm

Post Sat Jan 02, 2016 9:20 pm

Re: DB size growing

josch.hh wrote:
jhb50 wrote:
zip wrote:Online streams are not stored into the DB (Derby), but use a caching mechanism (Apache JCS). No idea how the space in the persistent cache is reclaimed... but i guess you already have better way to test it than me (I hardly use any online sources personally).

So online metadata is stored in C:\Program Files\Serviio\library\cache\online_technical_metadata.data (Mine is only 5MB) and the thumbs in and the thumbnails in C:\Program Files\Serviio\library\cache\thumbnails.data (Mine is 99MB). Files metadata is in C:\Program Files\Serviio\library\db (Mine is 282MB)


How many files of which kind are in your db?

Is there an easy way to get this number?
<<

zip

User avatar

Serviio developer / Site Admin

Posts: 17215

Joined: Sat Oct 24, 2009 12:24 pm

Location: London, UK

Post Sat Jan 02, 2016 9:28 pm

Re: DB size growing

jhb50 wrote:Is there an easy way to get this number?

You can see the numbers (images / audio / videos) in the new 1.6 console homepage.
<<

josch.hh

Serviio lover

Posts: 66

Joined: Sun Dec 20, 2015 10:01 pm

Post Sat Jan 02, 2016 11:13 pm

Re: DB size growing

zip wrote:
jhb50 wrote:Is there an easy way to get this number?

You can see the numbers (images / audio / videos) in the new 1.6 console homepage.


nice, so when can we expect this new version ? ;-)
Next

Return to Serviio Support & Help

Who is online

Users browsing this forum: No registered users and 24 guests

Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by ST Software for PTF.