Getting started: Installation

This document explains how to set up and run DetectMateLibrary for users and developers.

User setup

Purpose: install the library so you can import and use DetectMate in your projects.

Recommended: use the provided uv helper (bundled Python environment manager used in this repo). If you prefer pip/venv, create a virtualenv first.

Install the package in editable mode:

uv sync

Result: the package is installed into the active Python environment and changes to the source tree are reflected immediately.

To install it in a different venv as a library:

uv pip install --no-cache-dir <directory_detectmatelibrary>

Optional dependencies

Not all features require the same dependencies. DetectMateLibrary uses optional extras so you only install what you need: Optional Dependency Groups https://pydevtools.com/handbook/explanation/what-are-optional-dependencies-and-dependency-groups/

Extra	Installs	When you need it
`llm`	`openai`, `tenacity`, `scipy`, `scikit-learn`, `tiktoken`, `pandas`	Using the `LogBatcherParser` (LLM-based log parsing)
`dataframes`	`pandas`, `polars`	Using `EventDataFrame`, `ChunkedEventDataFrame`, or `DataNormalizer`
`polars-rtcompat`	`polars[rtcompat]`	Running on older CPUs without AVX2 support (e.g. some VMs or embedded hardware); not needed for standard deployments
`full`	`llm` + `dataframes` + `polars-rtcompat`	Installing every optional extra at once

Install an extra with uv sync:

uv sync --extra dataframes

Or with uv pip install / pip when installing as a library:

uv pip install "detectmatelibrary[dataframes]"
# or
pip install "detectmatelibrary[dataframes]"

Combine multiple extras if needed:

uv sync --extra dataframes --extra polars-rtcompat

Or install everything at once with the full extra:

uv sync --extra full
# or
uv pip install "detectmatelibrary[full]"

Developer setup

Purpose: prepare a development environment with test and lint tooling.

Step 1: Install Python development dependencies & pre-commit hooks

Install dev dependencies (testing, linters, formatters). The dev group also pulls in the full extra, so every optional dependency (LLM, dataframes, polars-rtcompat) is installed too:

uv sync --dev

Install pre-commit hooks (this repository uses prek to run pre-commit tooling):

uv run --dev prek install

Notes:

Ensure uv is available in PATH. If not, use your system Python + virtualenv and then uv sync --dev.
Run the pre-commit hooks locally with uv run --dev prek run -a before committing to catch style/typing issues early.

Step 2: Install Protobuf toolchain (only if you change proto files)

Purpose: compile .proto definitions into Python code.

Install protoc on Debian/Ubuntu:

sudo apt-get update
sudo apt-get install -y protobuf-compiler
protoc --version

Compile the project proto:

protoc \
  --proto_path=src/detectmatelibrary/schemas/ \
  --python_out=src/detectmatelibrary/schemas/ \
  src/detectmatelibrary/schemas/schemas.proto

Result: generated Python modules appear under src/detectmatelibrary/schemas/. If you edit proto files, re-run this command and commit generated code if required by your workflow.

Step 3: Run unit tests

The full test suite covers dataframe and LLM-parser code, so all extras must be present. Since uv sync --dev already installs the full extra, run all tests with:

uv run --dev pytest -s

Run tests with coverage (terminal summary):

uv run --dev pytest --cov=. --cov-report=term-missing

Tips:

Run a single test or directory to speed iteration: uv run --dev pytest tests/some_test.py::test_name -q

Troubleshooting

If uv is unavailable, use a Python virtualenv and the pip/pytest commands directly.
If protoc is missing, install the system package or download a prebuilt binary for your OS.
Always run commands from the project root so file paths (pyproject.toml, src/) resolve correctly.

Go back to Index or proceed to Basic usage.