C # FileSystemWatcher and FTP

Question

I keep track of files that are deleted on ftp through the file system watcher, then go to another directory. Now I am launching a copy from the file system observer creation event, but obviously in the case of ftp creation is just a stub, and the data arrives and populates the file as it is downloaded to completion. Does anyone have an elegant solution for this, or should I do what I think I need to do

1 wait till last access time is about n ms in past before I copy 2 throw a control file in there to state that that file is done being copied, then delete control file 3 pound the crap out of it 

Answer 1

This is a very naive implementation, but it is suitable for my purposes, I have seen enough people with this problem on the Internet, so I decided to contribute. The implementation is quite specific to my needs, I almost did not care about the changed events, given the nature of my problem, but people can throw their own code there if they need to do something else, this is really created, which causes most problems. I havent fully tested this, but first write that it looks good.

 using System; using System.Collections.Generic; using System.IO; using System.Timers; namespace FolderSyncing { public class FTPFileSystemWatcher { private readonly string _path; public event FileSystemEventHandler FTPFileCreated; public event FileSystemEventHandler FTPFileDeleted; public event FileSystemEventHandler FTPFileChanged; private Dictionary<string, LastWriteTime> _createdFilesToCheck; private readonly object _lockObject = new object(); private const int _milliSecondsSinceLastWrite = 5000; private const int _createdCheckTimerInterval = 2000; private readonly FileSystemWatcher _baseWatcher; public FTPFileSystemWatcher(string path, string Filter) { _path = path; _baseWatcher = new FileSystemWatcher(path,Filter); SetUpEventHandling(); } public FTPFileSystemWatcher(string path) { _path = path; _baseWatcher = new FileSystemWatcher(path); SetUpEventHandling(); } private void SetUpEventHandling() { _createdFilesToCheck = new Dictionary<string, LastWriteTime>(); Timer copyTimer = new Timer(_createdCheckTimerInterval); copyTimer.Elapsed += copyTimer_Elapsed; copyTimer.Enabled = true; copyTimer.Start(); _baseWatcher.EnableRaisingEvents = true; _baseWatcher.Created += _baseWatcher_Created; _baseWatcher.Deleted += _baseWatcher_Deleted; _baseWatcher.Changed += _baseWatcher_Changed; } void copyTimer_Elapsed(object sender, ElapsedEventArgs e) { lock (_lockObject) { Console.WriteLine("Checking : " + DateTime.Now); DateTime dateToCheck = DateTime.Now; List<string> toRemove = new List<string>(); foreach (KeyValuePair<string, LastWriteTime> fileToCopy in _createdFilesToCheck) { FileInfo fileToCheck = new FileInfo(_path + fileToCopy.Key); TimeSpan difference = fileToCheck.LastWriteTime - fileToCopy.Value.Date; fileToCopy.Value.Update(fileToCopy.Value.Date.AddMilliseconds(difference.TotalMilliseconds)); if (fileToCopy.Value.Date.AddMilliseconds(_milliSecondsSinceLastWrite) < dateToCheck) { FileSystemEventArgs args = new FileSystemEventArgs(WatcherChangeTypes.Created, _path, fileToCopy.Key); toRemove.Add(fileToCopy.Key); InvokeFTPFileCreated(args); } } foreach (string removal in toRemove) { _createdFilesToCheck.Remove(removal); } } } void _baseWatcher_Changed(object sender, FileSystemEventArgs e) { InvokeFTPFileChanged(e); } void _baseWatcher_Deleted(object sender, FileSystemEventArgs e) { InvokeFTPFileDeleted(e); } void _baseWatcher_Created(object sender, FileSystemEventArgs e) { if (!_createdFilesToCheck.ContainsKey(e.Name)) { FileInfo fileToCopy = new FileInfo(e.FullPath); _createdFilesToCheck.Add(e.Name,new LastWriteTime(fileToCopy.LastWriteTime)); } } private void InvokeFTPFileChanged(FileSystemEventArgs e) { FileSystemEventHandler Handler = FTPFileChanged; if (Handler != null) { Handler(this, e); } } private void InvokeFTPFileDeleted(FileSystemEventArgs e) { FileSystemEventHandler Handler = FTPFileDeleted; if (Handler != null) { Handler(this, e); } } private void InvokeFTPFileCreated(FileSystemEventArgs e) { FileSystemEventHandler Handler = FTPFileCreated; if (Handler != null) { Handler(this, e); } } } public class LastWriteTime { private DateTime _date; public DateTime Date { get { return _date; } } public LastWriteTime(DateTime date) { _date = date; } public void Update(DateTime dateTime) { _date = dateTime; } } } 

Answer 2

Wait until you can only open the file. I would not say that this is a good solution, but probably the safest strategy in the circumstances.

Answer 3

here is an implementation for synchronization.

 using System; using System.Configuration; using System.IO; using System.Threading; namespace FolderSyncing { public class FolderSync { private readonly string masterDirectoryPath; private readonly string slaveDirectoryPath; public FolderSync() { masterDirectoryPath = ConfigurationManager.AppSettings.Get("MasterDirectory"); slaveDirectoryPath = ConfigurationManager.AppSettings.Get("SlaveDirectory"); if (Directory.Exists(masterDirectoryPath) && Directory.Exists(slaveDirectoryPath)) { FTPFileSystemWatcher watcher = new FTPFileSystemWatcher(masterDirectoryPath); watcher.FTPFileChanged += watcher_FTPFileChanged; watcher.FTPFileCreated += watcher_FTPFileCreated; watcher.FTPFileDeleted += watcher_FTPFileDeleted; } else { Console.WriteLine("Directories were not located check config paths"); } } void watcher_FTPFileDeleted(object sender, FileSystemEventArgs e) { DeleteFile(slaveDirectoryPath + e.Name, 5, 1); } void watcher_FTPFileCreated(object sender, FileSystemEventArgs e) { CopyFile(e.Name,5,1); } void watcher_FTPFileChanged(object sender, FileSystemEventArgs e) { } private void DeleteFile(string fullPath, int attempts, int attemptNo) { if (File.Exists(fullPath)) { try { File.Delete(fullPath); Console.WriteLine("Deleted " + fullPath); } catch (Exception) { if (attempts > attemptNo) { Console.WriteLine("Failed deleting " + fullPath + "trying again "+ attemptNo.ToString()+ " of "+attempts); Thread.Sleep(500); DeleteFile(fullPath, attempts, attemptNo + 1); } else { Console.WriteLine("Critically Failed deleting " + fullPath); } } } } private void CopyFile(string fileName,int attempts, int attemptNo) { string masterFile = masterDirectoryPath + fileName; string slaveFile = slaveDirectoryPath + fileName; if (File.Exists(masterFile)) { try { File.Copy(masterFile,slaveFile); Console.WriteLine("Copied " + masterFile); } catch (Exception) { if (attempts > attemptNo) { Console.WriteLine("Failed copying " + masterFile + "trying again " + attemptNo.ToString() + " of " + attempts); Thread.Sleep(500); CopyFile(fileName, attempts, attemptNo + 1); } else { Console.WriteLine("Critically Failed copying " + masterFile); } } } } } } 

Answer 4

When a file is copied from another server using FTP, before the full copy of the file, the file name is renamed with an additional extension, for example .TMP, as shown in the example path below.

C: \ InterfaceServer \ OilAndGas \ XMLForTest \ TestStbFile.xml.TMP

To overcome this situation, follow two steps.

 1. When there is run of file watcher method, FileSystemEventArgs parameter contains file name with appended extra file extension to the file name as it just arrived in the folder and not completed with copy operation. 2. You need to just call the below method to remove the extra extension and add wait for 2 seconds in the code so that the complete file gets created and you can use it for further processing. public static string VerifyIfStubFile(string filePath, string extension) { if (filePath.Length - (filePath.IndexOf(extension) + extension.Length) != 0) { /*LogMsg("Info : VerifyIfStubFile : Stub file found. Lets fix it!!"); */ return filePath = filePath.Substring(0, (filePath.IndexOf(extension) + extension.Length)); } return filePath; } 

Answer 5

Download the source stub file immediately after completing the data file and ask the FileSystemWatcher file to view the stub file. For example, if the data file name is mydata_01234, then stub woulb will be mydata_01234_stub. Then FileSystemWatcher must have the mask "* _stub". Then you know the name of the data file by removing the suffix "_stub". And the stub file cannot be downloaded until the data file is complete, so the data file will be free.

If the stub files are just one byte, you can delete them after any operation that you perform on the data file without any problems. If your operations are especially fast, add 100 ms of sleep before removing the stub.

Answer 6

After 4 years....

a stub file is a good idea, but probably a slightly more reliable way to do this is to first create a stub source file, upload your “real” file, and then delete the stub.