In a previous article (Click to See Article), I described the process of creating a Windows Service that uses the FileSystemWatcher to track the movement of files on an FTP Server. In that article, I said that rare circumstances could cause the service application to go down and thereby cause us to lose the record of files being created. In that article, I stressed the requirement to contain as little business logic as possible in the service to reduce the amount of code that could possibly fail.
In this article, I will discuss the reading of the file created by the Windows Service application, and also the recovery methods that are in place to recover the track of files moved while the service is down on rare occassions. This article will demonstrate the use of several new methods found in the System.IO Namespace and at the same time demonstrate the methodology of recursively traversing multiple directories and their respective sub directories.
As you will recall, if you read the previous article, the Windows Service application runs on the FTP Server computer, creating a text file record of file movement. It simply records one line for each file created. That line contains the time of the creation and the name of the newly created file. The service application does not care what kind of file is being created, as that is the business of the application described by this article.
In this application, a Timer fires every five minutes to call the LoadFiles method, shown under Figure 1. This method attempts to rename the text file created by the Windows Service application. Once it is renamed successfully, it is read a line at a time, and each file and time is passed to a method called LoadOneFile. That method will not be shown here, but it is the method that determines whether a file is of interest and if it is, it is recorded in a SQL Server database. By renaming the file, that the service is writing to, we literally take that file away from the service, thereby causing it to create a new instance of the same filename.
Since both this application and the service application are real-time programs, they do not display message boxes. First, there is no one to answer the message, and second; we cannot allow the applications to stop. Therefore, we display error messages in a multi-line text box and try again the next time the timer fires. In some applications, when we encounter an error, we send emails to IT personnel, but for this article, I have omitted that code.
Figure 1 - LoadFiles Method.
| Private Overloads Sub LoadFiles() ' rename the current file that the file watcher service is writing to ' call LoadOneFile passing a single file to be loaded or discarded ' If successful in processing all of the files, mark the processed file ' and exit Dim i As Integer Dim start As Integer Dim RenameSuuccessful As Boolean = False Dim hIN As Integer Dim MyFile As String = "" Dim SQL As String Dim dt As New DataTable() Try DoEvents() If Dir(FWServiceFile).Length = 0 Then Exit Sub ' nothing to do, no files have been created End If ' clear the metrics With Me .txtLog.Text = "" .miActivityCounter = 0 .miSqlErrorCount = 0 End With ' try renaming the file MyFile = "I:\FileWatcherFiles\FW" & _ Format(Today, "MMddyyyy") & Format(Now, "HHmmss") & ".TXT" Do While Not RenameSuuccessful Try Rename(FWServiceFile, MyFile) RenameSuuccessful = True Catch start = Microsoft.VisualBasic.Timer Do While Microsoft.VisualBasic.Timer - start < 1<BR> DoEvents() Loop End Try Loop ' now the file is renamed, start loading to database ' get a handle and open the file Me.lblCurrentFile.Text = MyFile DoEvents() hIN = FreeFile() Try FileOpen(hIN, MyFile, OpenMode.Input, OpenAccess.Read) Catch ex As System.Exception Me.txtLog.AppendText("Err: " & ex.ToString & vbCrLf) Exit Sub End Try ' loop thru the file and load the files to database Dim dataLine As String Dim fileTime As String Dim fileName As String Dim trash As String Dim s() As String Do While Not EOF(hIN) dataLine = LineInput(hIN) If dataLine.Substring(0, 2) = "TM" Then s = dataLine.Split(vbTab) ' s(1) will be time string ' s(3) will be the file and path ' call the database recorder, it will record the ' creation of the file in database, only if the ' file is of an interesting type If Not LoadOneFile(s(3), s(1), Connection) Then Me.miSqlErrorCount += 1 Me.lblSqlErrCount.Text = miSqlErrorCount End If DoEvents() End If Loop Try FileClose(hIN) Catch ex As System.Exception Me.txtLog.AppendText("Err: " & ex.ToString & vbCrLf) End Try ' move the processed file to the archive. If the process ' does not get to here and the file is not moved, then it ' was not processed successfully... Try Dim archiveFile As String = Replace(MyFile, _ "I:\FileWatcherFiles", "I:\FileWatcherFiles\FWFArchive") System.IO.File.Move(MyFile, archiveFile) Catch ex As System.Exception Me.txtLog.AppendText("Err: " & ex.ToString & vbCrLf) End Try Catch ex As System.Exception Me.txtLog.AppendText("Err: " & ex.ToString & vbCrLf) End Try End Sub |
| ' trace recovery code Public Structure MissedFile Public Filename As String Public CreatedDate As DateTime End Structure Public colFiles As New Collection() Public dStart As DateTime Public reDateFolder As New _ Regex("^.*?\\(?<month>[1-9]|0[1-9]|1[0-2])-?(?<day>[1-9]" & _ "|0[1-9]|[1-3][0-9])-?(?<year>20[0-9]{2}|[0-1][0-9])(\\.*)?$") Public sReplacePattern As String = "${month}/${day}/${year}" |
| Private Sub TraverseDir(ByVal sDir As String) ' This is a recursive function that traverses ' a directory structure and finds files that are ' either not in date folders or are in date folders ' that are on or after the target date Dim sSubDirs As String() Dim s As String Dim d As DateTime Dim sR As String Try Try ' New .NET function for retrieving all sub directories ' for the top level directory (sDir) sSubDirs = Directory.GetDirectories(sDir) Catch es As System.Exception ' here we display a message box, because the recovery ' process is attended End Try For Each s In sSubDirs 'Force the date folder to be in a readable format sR = reDateFolder.Replace(s, MesReplacePattern).ToString 'If this path does contain a date folder, check to see ' if it is one of the dates we are looking for If sR.Length > 0 AndAlso reDateFolder.IsMatch(s) Then Try d = DateTime.Parse(sR) 'Go back one day from the start just to make sure If d >= dStart.AddDays(-1) Then 'Traverse this directory and check for missed files TraverseDir(s) GetMissedFiles(s) End If Catch ex As FormatException 'This is not a valid date - Traverse TraverseDir(s) GetMissedFiles(s) End Try Else 'There is no date folder - Traverse TraverseDir(s) GetMissedFiles(s) End If Next Catch ez As StopTracingDirectoryException Throw ez Catch ex As System.Exception MsgBox(ex.ToString) End Try End Sub Public Sub GetMissedFiles(ByVal sDir As String) 'This adds all files created after the given date 'in the given directory to a collection Dim mf As MissedFile Dim s As String Dim d As DateTime Dim sFiles As String() ' new .NET method for retrieving all of the files ' in the passed directory sFiles = Directory.GetFiles(sDir) For Each s In sFiles d = File.GetCreationTime(s) 'See if this file was created on or after our start date If d >= dStart Then 'If it is a new file, add it and its creation time to the collection mf = New MissedFile() mf.Filename = s mf.CreatedDate = d colFiles.Add(mf) End If Next End Sub |