Google Chronicle Logstash parsing: How to test your UDM parsing code

remington standard typewriter in greyscale photography

Google Chronicle is a cloud-based security information and event management (SIEM) platform developed by Google. It is designed to help organizations detect and investigate security threats in real-time by collecting and analyzing large volumes of security telemetry data from various sources, such as network traffic, endpoints, and cloud services. Google Chronicle provides advanced threat detection capabilities, machine learning-based analytics, and a user-friendly interface that enables security teams to quickly identify and respond to security incidents.

Logstash is an open-source data processing pipeline tool that is used for ingesting, processing, and transforming large volumes of data from various sources into a structured format. It is often used in conjunction with Elasticsearch and Kibana to form the ELK stack, which is a popular combination of tools used for centralized logging and data analysis. Logstash is highly configurable and supports a wide range of input and output plugins, making it a versatile tool for processing and analyzing data in real-time. It is commonly used in DevOps, IT operations, and security contexts to analyze logs and metrics data.

Problem

You have a parser code following the Logstash format, but you don’t know where you can test it.

Solution

I built an open-source Logstash parsing tool.

It’s very straight-forward: You simply plug in your raw log, and then your parser code.

Check it out here: https://chroniclelogparser.sapalo.dev/

Author’s note

I’m not really a security expert, or a logstash expert, but I’ve been trying to find a way to test logstash parsing code for raw logs, for parsing into Google Chronicle UDMs.

I don’t work in this field but I spent somewhere around 2 hours to set this whole system up. I think the UDM parsing might still be incorrect (70% vibe that it’s incorrect) but I think the prompt can be improved so that ChatGPT is taught about the UDM model.

I’m open to feedback, so please send over some GitHub issues if you find any.

Explore, Learn, and Thrive: Tech and Gaming with Darren

Hello, reader! I’m Darren, and I’m passionate about technology, learning, and gaming. My articles cater to mid to senior-level software engineers seeking to expand their knowledge and skills.

Through sharing our experiences and lessons learned (including our mistakes), we can inspire, support, and empower the next generation of engineering problem solvers. Documenting these insights also helps me reinforce their importance and ensure they remain in my memory.

In my blog, you’ll find a collection of mental models designed to help you tackle challenges in both your engineering career and personal life. Additionally, I share personal reflections and short stories, exploring parallels between competitive gaming and workplace performance.

Join me on this journey to learn and grow together!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.