After two years of stable online running, we decided to open the kids!

As mentioned in the previous article, kids is a log aggregation system. As we all know, log is the gold mine of every Internet company, so it is one of the most important systems. It is known that all logs are aggregated and stored through this system.

How do we use it?

In Zhihu, we have deployed kids on all our machines and configured it into agent mode, which is responsible for forwarding the logs generated by the native application to the upstream kids. There will be a central server at the upstream, which is configured with a persistent store to be responsible for persistence. Of course, the most upstream server is also the place of the largest message subscription. The following picture:

Details of kids implementation

In terms of aggregation, kids implements the design of scribe, but the XML configuration file is really unfriendly, so it uses ragel and lemon to write a cleaner configuration file.

In terms of pub / sub, we migrated the implementation of redis, so that we realized the storage function of redis pub / sub combined with scribe.

In the thread model, we adopt the mode of master thread receiving connection and worker thread processing. The structure is as follows:

All client connections are received by the master thread, and the master then evenly distributes the connections to each processing thread. After that, all commands of the client are processed by the worker thread.

All worker threads share a global queue, each worker maintains its own queue pointer, and the master has a scheduled task to release messages that have been consumed by all consumers.

At the same time, there is a separate IO thread to handle store and forward related io. This can make full use of multi-core parallelism.

All threads use socketpair to notify asynchronously to minimize synchronous calls and increase throughput. The only synchronous call left in kids is to write disk.

After open source

It is only the first step of open source to open the code, which brings more possibilities and greater vitality to the project of kids. We will continue to maintain it and push it to be simpler, more stable and more practical.

In the short term, we will strengthen the monitoring improvement. And enhance the reliability of log forwarding under high load.

If you have other ideas, submit issue or pull request: -)

Want to try? Here's the code: Zhihu / kids · GitHub