-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ConcurrentWrites with atomic file writes on .Net5 platform #4244
Comments
Looks like FileStream-constructor with FileSystemRights is available: dotnet/runtime#30435 |
But it looks like it requires a dependency: https://www.nuget.org/packages/System.IO.FileSystem.AccessControl/ Guess it has to be implemented in a seperate Nuget-package. Maybe fix FileTarget so it is easy override the creation of FileStream-object. Then create new ConcurrentFileTarget-nuget-package with support for atomic-writes. |
Consider creating this a ConcurrentFileTarget-nuget-package that depends on these packages for Net5: https://www.nuget.org/packages/System.IO.FileSystem.AccessControl/ And then adjust the vanilla FileTarget so it is easy to override FileStream-handling and Mutex-handling where necessary. Maybe the same nuget-package could add support for assigning File-Permissions on file-creation (nice-to-have) |
Thinking that FileTarget should have 2 methods that ConcurrentFileTarget could override:
Instead of FileTarget working with different FileAppenders, then it should just have two types. One for KeepFileOpen=false (Contains filename) and one for KeepFileOpen=true (Contains filename + stream) The ConcurrentFileTarget can then make a special Stream, that on dispose will also dispose any mutex-object if necessary. Then the Stream-object become the wrapper for any special writing-behavior (ex. writing in gzip format). The ConcurrentFileTarget can on Something like: protected override void ArchiveFileStream(Stream openStream, string currentFilename, string archiveFilename)
{
using (var mutexProtection = ExtractArchiveMutex(openStream))
{
base.ArchiveFileStream(openStream, currentFilename, archiveFilename);
}
} |
See also: dotnet/runtime#53432 |
I use the global mutex for concurrent writes and create it in advance with the everyone permission that the .Net 6 modules work together with .Net Framework 4.8 modules. I saw that there is a new feature for atomic writes in .Net which could be used instead of the mutex. |
The pretty solution would be a refactoring of NLog FileTarget, so one can override the FileAppender-factory. So NLog FileTarget gets a virtual protected method that returns a Then one can create a new nuget-package that extends the NLog FileTarget and overrides the protected method, and then using p/invoke to call the Win32-API Method-CreateFile and provides the magic parameters that enables atomic-file-appending for the file-stream. Anyway just a random idea, where one skips the use of mutex. |
I have a question regarding the File Archiver Mutex. My Workaround for .Net 4.8 and .Net 6 to write to the same logfile is to create the file and file archive mutex manually before I initialize NLog. Then the Mutex Security is correct even for .Net 6 modules. In some rare cases I still have problems with mutex security and then we have to stop all applications that write to that file otherwise the mutex remains and some applications are not able to write to the log file and are completely blocked. Could it be that NLog recreates the File Archive Mutex at some point? |
Believe NLog re-creates mutex object after each archive (or detecting if file is created /deleted in file logging folder).
But NLog should always handle, that mutex already exists. I'm guessing that the mutex is only completely "released"/"cleared" when all mutex objects are disposed.
Can you share how you perform early mutex initializing with NET6? (I'm guessing your app keeps this initial mutex object alive the entire app lifetime)
|
I use the same code as NLog except for the Mutex creation itself. public static void checkSharableMutex(String filename)
{
if (System.Runtime.InteropServices.RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
{
CreateSharableMutex("FileArchiveLock", filename);
CreateSharableMutex("FileLock", filename);
}
}
private static Mutex CreateSharableMutex(string mutexNamePrefix, string filename)
{
var name = GetMutexName(mutexNamePrefix, filename);
return ForceCreateSharableMutex(name);
}
private static Mutex ForceCreateSharableMutex(string name)
{
if (System.Runtime.InteropServices.RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
{
// Creates a mutex sharable by more than one process
var mutexSecurity = new MutexSecurity();
var everyoneSid = new SecurityIdentifier(WellKnownSidType.WorldSid, null);
mutexSecurity.AddAccessRule(new MutexAccessRule(everyoneSid, MutexRights.FullControl, AccessControlType.Allow));
#if NET48
// The constructor will either create new mutex or open
// an existing one, in a thread-safe manner
return new Mutex(false, name, out _, mutexSecurity);
#else
return MutexAcl.Create(false, name, out _, mutexSecurity);
#endif
}
return new Mutex(false, name, out _);
} |
Think you should keep the Mutex-objects alive, so the Mutex-creation is not "released"/"cleared" unexpectedly at garbage collection: private static Mutex _fileLock;
private static Mutex _fileArchiveLock;
public static void checkSharableMutex(String filename)
{
if (System.Runtime.InteropServices.RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
{
_fileLock = CreateSharableMutex("FileLock", filename);
_fileArchiveLock = CreateSharableMutex("FileArchiveLock", filename);
}
} |
Hi @snakefoot, I was wondering if you would add more detail to the following comment:
Why The downside is adding |
NLog v6 will extract FileTarget into its own nuget-package, and then everything is possible. NLog v5 can open doors on FileTarget, so one can create an extension nuget-package, that adds support for atomic-file-append on Windows / Linux along with correct mutex-handling. |
.Net5 will automatically use the NetStandard2-dependency, so ConcurrentWrites will depend on global mutex (And the global mutex can only be used between applications using the same service-account).
Could be nice if support for atomic-file-appending could be enabled for the .Net5-platform. And the global-mutex could be used across service-accounts just like on .NetFramework.
The text was updated successfully, but these errors were encountered: