I'm looking for a flexible event logging platform to store both pre-defined (username, ip address) and non-pre-defined (can be generated as needed by any piece of code) events for Django. I'm currently doing some of this with log files, but it ends up requiring various analysis scripts and ends up in a DB anyway, so I'm considering throwing it immediately into a nosql store such as MongoDB or Redis.

The idea is to be easily able to query, for example, which ip address the user most commonly comes from, whether the user has ever performed some action, lookup the outcome for a specific event, etc.

Is there something that already does this?

If not, I'm thinking of this:

The "event" is a dictionary attached to the request object. Middleware fills in various pieces (username, ip, sql timing), code fills in the rest as needed.

After the request is served a post-request hook drops the event into mongodb/redis, normalizing various fields (eg. incrementing the username:ip address counter) and dropping the rest in as is.

Words of wisdom / pointers to code that does some/all of this would be appreciated.

score:0

This question was done years ago.

However, anyone that are searching for the same issue may get some help from this answer.

Basically Django has a LogEntry model that can handle all the user's logs from the admin app:

from django.admin.models import LogEntry

For example, you can easily use this to create an endpoint using django-rest-framework to consume the log-data, just adding this models as Meta.model = LogEntry and querysets = LogEntry.objects.all() for viewsets.

To connect with mongodb in django, today there is a greate package called djongo.

pip install djongo

and in settings.py you can add the following code:

# database setup to connect with mongodb.
DATABASES = {
    'default': {
        'ENGINE': 'djongo',
        'NAME': 'demodatabase',
        'CLIENT': {
           'host': 'your-db-host',
        }
    }
}

Here is one example of a log:

    {
        "id": 3,
        "action_time": "2021-09-24T17:19:51.252000Z",
        "object_id": "1",
        "object_repr": "Test title",
        "action_flag": 1,
        "change_message": "[{\"added\": {}}]",
        "user": 1,
        "content_type": 7
    },

You can also add more than one database, like adding only mongodb to receive logs you will just need to create routers to handle which database do what.

You can have a look at the documentation where you can get a better explanation.

score:1

I'm not sure if a library exists that does exactly what you're looking for with a NoSQL DB. However, a MongoDB backend was just released and it appears to work with Django's ORM (in a limited fashion).

Here's the link.

Perhaps you could use this with an existing logging system that utilizes Django's built-in ORM? I hope this helps, if only a little.

score:3

The mongodb-log project can probably be used as a foundation for what you want to do.


Related Query