1. Demand
The requirement is simple, that is, to write logs at high speed in C# development. For example, in high concurrency, high traffic places need to write logs. We know that programs are time-consuming in operating disks, so we will spend some time writing logs to disks, which is not what we want to see.
2. Solutions
2.1. Simple Principle Explanation
Using queues, we first cache to memory, and then we have a thread that writes to disk from the queue, so that we can write logs with high speed and high performance. Because of the slow speed, we separated it, that is to say, after the program throws the log to the queue, the log part of the program is completed, and the later part of the program that takes time to operate the disk is not concerned, and it is operated by another thread.
As the saying goes, both fish and bear's paw can not be obtained, so there will be a problem, that is, if the log has arrived in the queue at this time, program crash or computer power failure will lead to partial loss of the log, but some places in order to write a high-performance log, whether you can ignore some cases, please depend on the situation.
2.2. Examples
3. Key Code Section
The part of LZ that writes logs here is chosen to be more commonly used. log4net Of course, you can also choose other log components, such as nlog and so on.
3.1. Log to Queue
In the first step, we need to put the log in the queue first, then we can write it to disk from the queue.
public void EnqueueMessage(string message, FlashLogLevel level, Exception ex = null) { if ((level == FlashLogLevel.Debug && _log.IsDebugEnabled) || (level == FlashLogLevel.Error && _log.IsErrorEnabled) || (level == FlashLogLevel.Fatal && _log.IsFatalEnabled) || (level == FlashLogLevel.Info && _log.IsInfoEnabled) || (level == FlashLogLevel.Warn && _log.IsWarnEnabled)) { _que.Enqueue(new FlashLogMessage { Message = "[" + DateTime.Now.ToString("yyyy-MM-dd HH:mm:ss,fff") + "]\r\n" + message, Level = level, Exception = ex }); // Notify threads to write logs to disk _mre.Set(); } }
_ Log is the ILog of log4net log component, which includes the functions of writing logs and judging the log level. If judging at the beginning of the code is to compare the judgment level with the current log level and see if it needs to be written to the queue. This can effectively improve the performance of the log.
Where _que is the Concurrent Queue queue. _ mre is a Manual Research Event signal, and Manual Research Event is used to notify thread queues of new logs that can be written to disk from the queue. When the log is written from the queue, reset the signal and wait for the next new log to arrive.
3.2. Queuing to Disk
From queue to disk, we need a thread to write from queue to disk, that is to say, we need to load this thread when the program starts, such as asp.net in the Application_Start in global.
/// <summary> /// Another thread logs only once when the program is initialized /// </summary> public void Register() { Thread t = new Thread(new ThreadStart(WriteLog)); t.IsBackground = false; t.Start(); } /// <summary> /// Write logs from queues to disks /// </summary> private void WriteLog() { while (true) { // Waiting for signal notification _mre.WaitOne(); // Determine if there is a need for content such as disk while (_que.Count > 0) { FlashLogMessage msg; if (_que.TryDequeue(out msg)) // Get the content from the queue and delete the content from the queue { // Judge the log level, and then write the log switch (msg.Level) { case FlashLogLevel.Debug: _log.Debug(msg.Message, msg.Exception); break; case FlashLogLevel.Info: _log.Info(msg.Message, msg.Exception); break; case FlashLogLevel.Error: _log.Error(msg.Message, msg.Exception); break; case FlashLogLevel.Warn: _log.Warn(msg.Message, msg.Exception); break; case FlashLogLevel.Fatal: _log.Fatal(msg.Message, msg.Exception); break; } } } // Reset the signal _mre.Reset(); } }
3.3. Complete code
using log4net; using log4net.Config; using System; using System.Collections.Concurrent; using System.IO; using System.Threading; namespace Emrys.FlashLog { public sealed class FlashLogger { /// <summary> /// Record messages Queue /// </summary> private readonly ConcurrentQueue<FlashLogMessage> _que; /// <summary> /// signal /// </summary> private readonly ManualResetEvent _mre; /// <summary> /// Journal /// </summary> private readonly ILog _log; /// <summary> /// Journal /// </summary> private static FlashLogger _flashLog = new FlashLogger(); private FlashLogger() { // Setting Log Profile Path XmlConfigurator.Configure(new FileInfo(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "log4net.config"))); _que = new ConcurrentQueue<FlashLogMessage>(); _mre = new ManualResetEvent(false); _log = LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType); } /// <summary> /// Implementing singletons /// </summary> /// <returns></returns> public static FlashLogger Instance() { return _flashLog; } /// <summary> /// Another thread logs only once when the program is initialized /// </summary> public void Register() { Thread t = new Thread(new ThreadStart(WriteLog)); t.IsBackground = false; t.Start(); } /// <summary> /// Write logs from queues to disks /// </summary> private void WriteLog() { while (true) { // Waiting for signal notification _mre.WaitOne(); // Determine if there is a need for content such as disk while (_que.Count > 0) { FlashLogMessage msg; if (_que.TryDequeue(out msg)) // Get the content from the queue and delete the content from the queue { // Judge the log level, and then write the log switch (msg.Level) { case FlashLogLevel.Debug: _log.Debug(msg.Message, msg.Exception); break; case FlashLogLevel.Info: _log.Info(msg.Message, msg.Exception); break; case FlashLogLevel.Error: _log.Error(msg.Message, msg.Exception); break; case FlashLogLevel.Warn: _log.Warn(msg.Message, msg.Exception); break; case FlashLogLevel.Fatal: _log.Fatal(msg.Message, msg.Exception); break; } } } // Reset the signal _mre.Reset(); } } /// <summary> /// Write a log /// </summary> /// <param name="message">Log text</param> /// <param name="level">Grade</param> /// <param name="ex">Exception</param> public void EnqueueMessage(string message, FlashLogLevel level, Exception ex = null) { if ((level == FlashLogLevel.Debug && _log.IsDebugEnabled) || (level == FlashLogLevel.Error && _log.IsErrorEnabled) || (level == FlashLogLevel.Fatal && _log.IsFatalEnabled) || (level == FlashLogLevel.Info && _log.IsInfoEnabled) || (level == FlashLogLevel.Warn && _log.IsWarnEnabled)) { _que.Enqueue(new FlashLogMessage { Message = "[" + DateTime.Now.ToString("yyyy-MM-dd HH:mm:ss,fff") + "]\r\n" + message, Level = level, Exception = ex }); // Notify threads to write logs to disk _mre.Set(); } } public static void Debug(string msg, Exception ex = null) { Instance().EnqueueMessage(msg, FlashLogLevel.Debug, ex); } public static void Error(string msg, Exception ex = null) { Instance().EnqueueMessage(msg, FlashLogLevel.Error, ex); } public static void Fatal(string msg, Exception ex = null) { Instance().EnqueueMessage(msg, FlashLogLevel.Fatal, ex); } public static void Info(string msg, Exception ex = null) { Instance().EnqueueMessage(msg, FlashLogLevel.Info, ex); } public static void Warn(string msg, Exception ex = null) { Instance().EnqueueMessage(msg, FlashLogLevel.Warn, ex); } } /// <summary> /// Log Level /// </summary> public enum FlashLogLevel { Debug, Info, Error, Warn, Fatal } /// <summary> /// Log content /// </summary> public class FlashLogMessage { public string Message { get; set; } public FlashLogLevel Level { get; set; } public Exception Exception { get; set; } } }
4. Performance comparison and Application
4.1. Performance comparison
After testing, it was found that
Using the original log4net to write 100,000 pieces of data to the log requires 29,178 milliseconds.
It takes only 261 milliseconds for the same data to be queued.
4.2. Application
4.2.1. Registration is required at program startup, such as Application_Start registration in Global.asax in asp.net program.
public class MvcApplication : System.Web.HttpApplication { protected void Application_Start() { AreaRegistration.RegisterAllAreas(); FilterConfig.RegisterGlobalFilters(GlobalFilters.Filters); RouteConfig.RegisterRoutes(RouteTable.Routes); BundleConfig.RegisterBundles(BundleTable.Bundles); FlashLogger.Instance().Register(); } }
4.2.2. Call FlashLogger's static method directly where you need to write to the log.
FlashLogger.Debug("Debug"); FlashLogger.Debug("Debug", new Exception("testexception")); FlashLogger.Info("Info"); FlashLogger.Fatal("Fatal"); FlashLogger.Error("Error"); FlashLogger.Warn("Warn", new Exception("testexception"));
5. Code Open Source
https://github.com/Emrys5/Emrys.FlashLog
Finally, I hope it will be helpful to you. This article is original. Welcome to make bricks and recommendation.