FAQ  •  Register  •  Login

Metadata Lookup Failures. Force rescan specified files only

<<

mike.clayton999

Streaming enthusiast

Posts: 43

Joined: Tue Feb 08, 2011 1:30 pm

Post Sat Oct 29, 2011 7:49 am

Metadata Lookup Failures. Force rescan specified files only

This VBs Script (Windows Only) will parse the Serviio.log file looking for films which failed online Metadata lookup with an error... Ie time out/Connection error while querying the online database and will Move them to another folder so they can be reimported to force Metadata Rescan.

Any film found with MetaData lookup errors will be moved from it's current location to <Film Drive>\Temp\Serviio\<subfolders> (Keeping your folder structure). With the films moved to a new folder Serviio will remove the films from the database (about an hour). Once the films have been removed from the database manually copy the films back to their original location to get Serviio to rescan Metadata for just these films.

When run the script will ask for the root path to your films.. IE x:\myfilms it uses this in conjunction with Serviio.log to identify series folders so it can keep the folder structure. Each film with a lookup error will be moved to <Film Drive>\temp\Serviio\<Sub Folders>

This script is designed to enable you to rescan just films which got metadata Errors, so you don't have to rescan all the files and maybe get lookup errors on other films.


Just copy the code into NotePad and save to c:\program files\Serviio_DLNA\Log\MetaDataFileMove.vbs (Must be in the same folder as Serviio.log)

  Code:
'**************************************
' MetaData Error File Move
'**************************************
' This script will move file with Metadata
' lookup errors out of the serviio folder
' and into \temp\serviio so they can be
' moved back to force Serviio to rescan
'**************************************
' Written By Mike Clayton
' Version 1.1
' **************************************
' 1.0   Original
' 1.1   Made script more generic for other people to use
'**************************************

On Error Resume Next
Dim WshShell,FSO,strFilmRoot,strFilmRootDrive
const ForReading =1
const ForWriting =2
Set WshShell = CreateObject("Wscript.shell")
Set fso = CreateObject ("scripting.filesystemobject")

GetFilmRoot()      ' Get Root of movie dir (so Episodes can be copied)
CreateTempFolder()   ' Create c:\temp\Serviio


' Open Serviio Log file
set Input= fso.openTextFile ("Serviio.log",ForReading , False)
IF Err.Number <> 0 Then
   Wscript.echo "ERROR - Unable to open Serviio Log file"
   Wscript.Quit
End If

Do while not Input.AtEndOfStream

   strInput = Input.ReadLine
   
   'Check if MetaData error Detected.
   If Instr(1,strInput,"WARN  [LibraryManager] Cannot read metadata",1) Then
      'Chop off everything after the end of the film name
      strInput = Left (strInput,Instr(strInput,"via extractor ONLINE_VIDEO_SOURCES") -1)

      'Chop out the Film Name
      strFileName = Mid (strInput, instrrev(strInput,"\") +1)

      'Chop out the Film Path
      strPath = Trim(Replace (UCase(Mid (strInput ,Instr(strInput,":\")-1)), Ucase(strFileName),""))

      If strPath = strFilmRoot Then
         ' Film not an Episode
         Err.Clear
         fso.MoveFile strPath & strFileName , strFilmRootDrive & "\temp\serviio" & "\"
      Else
         ' Episode not a Film
         strSubFolder = Mid (strPath ,instrRev(strPath,"\",len(strPath)-1) +1)
         If fso.FolderExists (strFilmRootDrive & "\temp\Serviio\" & strSubFolder) Then
            'Temp Folder structure exists
         Else
            'Create subfolder (Episodes)
            oFolder = fso.CreateFolder(strFilmRootDrive & "\temp\Serviio\" & strSubFolder)
         End If
         Err.Clear
         fso.MoveFile strPath & strFileName , strFilmRootDrive & "\temp\serviio\" & strSubFolder
      End If
         
      IF Err.Number <> 0 Then
         Wscript.echo "ERROR Film " & strFileName &" wasn't moved"
         Err.Clear
      End if
   End If
Loop
   
strResponse = MsgBox ("All Done !." & vbcrlf & "Films with Metadata errors are now here " & vbcrlf & vbcrlf & _
   strFilmRootDrive & "\Temp\Serviio" & vbcrlf & vbcrlf & "Wait for serviio to remove the files from the database" & vbcrlf & "and then Copy them back so Serviio will rescan them", _
   VBOKOnly + VBInformation ,"All Done...")
   
   
' Create folder structure to move film to
Function CreateTempFolder()
   On Error Resume Next
   If fso.FolderExists (strFilmRootDrive & "\temp") Then
      ' C:\temp Exists
   Else
      Set oFolder = fso.CreateFolder (strFilmRootDrive & "\temp")
   End IF
   
   If fso.FolderExists (strFilmRootDrive & "\temp\Serviio") Then
      ' c:\temp\serviio Exists
   Else
      Set oFolder = fso.CreateFolder (strFilmRootDrive & "\temp\Serviio")
   End IF
End Function
   
' Get full path to the film root folder
Function GetFilmRoot ()
   On Error Resume Next
   Do
      strFilmRoot = UCase(Trim(InputBox ("Enter the full path to your Film Directory / Or Quit","Film Path",strFilmRoot))) &"\"
      If strFilmRoot = "QUIT\" Then
         Wscript.quit()
      End If
      IF mid(strFilmRoot,2,1) <> ":" Then
         strReturn = MsgBox ("Please include the Drive letter in the path",VBOKOnly + VBCritical, "Error no Drive Letter")
         strFilmRoot = "Invalid Path"
      End If
      
      If fso.FolderExists (strFilmRoot) Then
         ' Valid folder path entered
         bValidDir = True
      Else
         ' Invalid path entered.  Loop to allow reenter
         strReturn = MsgBox ("Unable to locate the directory you speicified ",VBOKOnly + VBCritical, "Invalid Path")
         bValidDir = False
         strFilmRoot = "Invalid Path"
      End If
   Loop until bValidDir
   strFilmRootDrive = Left (strFilmRoot,2)
End Function
      
<<

Illico

User avatar

DLNA master

Posts: 4646

Joined: Fri Jul 23, 2010 8:08 am

Location: France

Post Wed Nov 02, 2011 9:28 am

Re: Metadata Lookup Failures. Force rescan specified files o

Thanks for that script,
we probably have to install some windows packages, no ?
Microsoft Visual Studio, etc ...
Illico
Sony 2010 KDL-46EX501 | BDP-S373 | SMP-N100 | Windows 8-i7 | Network DD HD-CE1.5TLU2 | Livebox2-UHD86 | ServiiDroid | ServiiGo | BubbleUPnP
Serviio beta tester - Moderator
<<

moltra

DLNA master

Posts: 1871

Joined: Thu Mar 24, 2011 11:00 pm

Location: Ohio USA

Post Wed Nov 02, 2011 11:01 am

Re: Metadata Lookup Failures. Force rescan specified files o

One thing that I would like to recommend is a log or display of the files that had metadata errors. The errors could be caused by a file name being incorrect compared to what is in the metadata database.

Another way to let the user know to check the file names is a msgbox at the end telling the user to verify the file names to the database to ensure that they are found the next time before copying back to serviio.

Other than this the script is great and will be a huge help to everyone.
Mark
Beta Tester Group
http://www.serviidb.com Online media resource repository

Netgear EVA2000 | Samsung BD-D5300 | XBOX 360 | Windows 7 | Mint Debian 12 | Raxz Maxx
HowTo: Provide supported formats of a device HowTo: Record a new ticket on Bitbucket
HowTo: Provide details of a video file that doesn't play HowTo: Turn on detailed logging
<<

mike.clayton999

Streaming enthusiast

Posts: 43

Joined: Tue Feb 08, 2011 1:30 pm

Post Fri Feb 10, 2012 9:00 pm

Re: Metadata Lookup Failures. Force rescan specified files o

Here you go. Updated code which gives you the option to list affected movies as well as move them. Also added a second log file which lists lookup failures as well as metadata timeouts.

Just save the script as MetaDataLookupErrors.vbs in the same folder as your serviio logs (Normally C:\Program Files\Serviio\Logs) then just double click on the VBS. This version works on WIndows Only (Sorry VBScript Code)



  Code:
'**************************************
' MetaData Error File Move
'**************************************
' This script will move file with Metadata
' lookup errors out of the serviio folder
' and into \temp\serviio so they can be
' moved back to force Serviio to rescan
'**************************************
' Written By Mike Clayton
' Version 1.2
' **************************************
' 1.0   Original
' 1.1   Made script more generic for other people to use
' 1.2   Add Log Only mode option
'**************************************

On Error Resume Next
Dim WshShell,FSO,strFilmRoot,strFilmRootDrive,bListOrMove
const ForReading =1
const ForWriting =2
Set WshShell = CreateObject("Wscript.shell")
Set FSO = CreateObject ("scripting.filesystemobject")

ListOrMove()         ' List Files with errors or Move them.
GetFilmRoot()          ' Get Root of movie dir (so Episodes can be copied)
CreateTempFolder()   ' Create <FilmDrive>:\temp\Serviio


' Open Serviio Log file
set Input= FSO.openTextFile ("Serviio.log",ForReading , False)
If Err.Number <> 0 Then
   Wscript.echo "ERROR - Unable to open Serviio Log file"
   Wscript.Quit
End If

' Open Metadata Log file
Set Output=FSO.OpenTextFile (strFilmRootDrive & "\Temp\MetaDataLog.txt",ForWriting ,True)
If Err.Number <> 0 Then
   Wscript.echo "ERROR - Unable to open Metadata Log file"
   Wscript.Quit
Else
   Output.WriteLine vbCrLf & "Metadata Timeout Warnings (Time out error while querying Database)" & vbCrLf & vbCrLf
End If

' Open Metadata Log file
Set Output1=FSO.OpenTextFile (strFilmRootDrive & "\Temp\MetaDataNoResult.txt",ForWriting ,True)
If Err.Number <> 0 Then
   Wscript.echo "ERROR - Unable to open MetadataNoResult Log file"
   Wscript.Quit
Else
   Output1.WriteLine vbCrLf & "Metadata Lookup Warnings (No File name match in Database)" & vbCrLf & vbCrLf
End If

Do while not Input.AtEndOfStream

   strInput = Input.ReadLine

   ' Online Lookup Error
   IF Instr(1,strInput,"WARN  [OnlineVideoSourcesMetadataExtractor] Online metadata search returned no results",1) > 0 Then
   OutPut1.WriteLine Mid (strInput , 120 , instr(120,strInput,"[VideoDescription",1)-121)
   End If

   'Check if MetaData Timeout error Detected.
   If (InStr(1,strInput,"WARN  [LibraryManager] Cannot read metadata",1) > 0) And (Instr(1,strInput,"via extractor ONLINE_VIDEO_SOURCES",1) > 0) Then
      'Chop off everything after the end of the film name
      strInput = Left (strInput,Instr(strInput,"via extractor ONLINE_VIDEO_SOURCES") -1)
     
      'Chop out the Film Name
      strFileName = Mid (strInput, instrrev(strInput,"\") +1)

      'Chop out the Film Path
      strPath = Trim(Replace (UCase(Mid (strInput ,Instr(strInput,":\")-1)), Ucase(strFileName),""))

      If strPath = strFilmRoot Then
         ' Film not an Episode
         Output.WriteLine strPath & strFileName
         Err.Clear
         If bListOrMove = vbYes Then
            fso.MoveFile strPath & strFileName , strFilmRootDrive & "\temp\serviio" & "\"
         End If
      Else
         ' Episode not a Film
         Output.WriteLine strPath & strFileName
         Err.Clear
         If bListOrMove = vbYes Then
            strSubFolder = Mid (strPath ,instrRev(strPath,"\",len(strPath)-1) +1)
            If fso.FolderExists (strFilmRootDrive & "\temp\Serviio\" & strSubFolder) Then
               'Temp Folder structure exists
            Else
               'Create subfolder (Episodes)
               oFolder = fso.CreateFolder(strFilmRootDrive & "\temp\Serviio\" & strSubFolder)
            End If
                 
            Err.Clear
            fso.MoveFile strPath & strFileName , strFilmRootDrive & "\temp\serviio\" & strSubFolder
         End If
      End If
         
      IF Err.Number <> 0 Then
         Wscript.echo "ERROR Film " & strFileName &" wasn't moved"
         Err.Clear
      End if
   End If
Loop

'Close Logfiles and copy to one.
Input.Close
Output.Close
Output1.Close

If bListOrMove = vbYes Then   
   strResponse = MsgBox ("All Done !." & vbcrlf & "Films with Metadata errors are now here " & vbcrlf & vbcrlf & _
   strFilmRootDrive & "\Temp\Serviio" & vbcrlf & vbcrlf & "Wait for serviio to remove the files from the database" & vbcrlf & "and then Copy them back so Serviio will rescan them", _
   VBOKOnly + VBInformation ,"All Done...")
Else
   strResponse = MsgBox ("All Done !." & vbcrlf & "Films with Metadata errors are now listed here" & vbcrlf & vbcrlf & _
   strFilmRootDrive & "\Temp\MetaDataLog.txt" & vbCrLf & strFilmRootDrive & "\Temp\MetaDataNoResult.txt" &  vbCrLf & vbCrLf & "Press OK to open in your default txt editor"  , VBOKCancel + VBInformation ,"All Done...")
   
   If strResponse = vbOK Then
      If FSO.FileExists (strFilmRootDrive & "\Temp\MetaDataLog.txt") Then
         WshShell.Run strFilmRootDrive & "\Temp\MetaDataLog.txt",1,False
      Else
         strResponse = WshShell.Popup ("No Metadata Timeout Warnings", 10, "No Timeout Warnings", vbOKOnly + vbinformation)
      End If
      If FSO.FileExists (strFilmRootDrive & "\Temp\MetaDataNoResult.txt") Then
         WshShell.Run strFilmRootDrive & "\Temp\MetaDataNoResult.txt",1,False
      Else
         strResponse = WshShell.Popup ("No Metadata Lookup Warnings", 10, "No Lookup Warnings", vbOKOnly + vbinformation)
      End If
   End If
End If
 
' Get user input to List files with MetaData Errors or Move them
Function ListOrMove()
   On Error Resume Next
   bListOrMove = MsgBox ("Do you want to Move or List files with MetaData Warnings" & vbCrLf & _
   vbCrLf & vbCrLf & "Yes = Move" & vbCrLf & "No = List",vbYesNo ,"List Or Move Files")
End Function
   
' Create folder structure to move film to
Function CreateTempFolder()
   On Error Resume Next
   If fso.FolderExists (strFilmRootDrive & "\temp") Then
      ' <FilmRoot>\temp Exists
   Else
      Set oFolder = fso.CreateFolder (strFilmRootDrive & "\temp")
   End IF
   
   If fso.FolderExists (strFilmRootDrive & "\temp\Serviio") Then
      ' <FilmRoot>\temp\serviio Exists
   Else
      Set oFolder = fso.CreateFolder (strFilmRootDrive & "\temp\Serviio")
   End IF
End Function
   
' Get full path to the film root folder
Function GetFilmRoot ()
   On Error Resume Next
   Do
      strFilmRoot = UCase(Trim(InputBox ("Enter the full path to your Film Directory / Or Quit","Film Path",strFilmRoot))) &"\"
      If strFilmRoot = "QUIT\" Then
         Wscript.quit()
      End If
      IF mid(strFilmRoot,2,1) <> ":" Then
         strReturn = MsgBox ("Please include the Drive letter in the path",VBOKOnly + VBCritical, "Error no Drive Letter")
         strFilmRoot = "Invalid Path"
      End If
     
      If fso.FolderExists (strFilmRoot) Then
         ' Valid folder path entered
         bValidDir = True
      Else
         ' Invalid path entered.  Loop to allow reenter
         strReturn = MsgBox ("Unable to locate the directory you speicified ",VBOKOnly + VBCritical, "Invalid Path")
         bValidDir = False
         strFilmRoot = "Invalid Path"
      End If
   Loop until bValidDir
   strFilmRootDrive = Left (strFilmRoot,2)
End Function

Return to Third-party tools integration

Who is online

Users browsing this forum: No registered users and 17 guests

Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by ST Software for PTF.