Skip to content

Conversation

@tomaz-lc
Copy link
Contributor

@tomaz-lc tomaz-lc commented Jan 29, 2026

Description

Add limacharlie usp validate CLI command for testing USP adapter parsing configurations against sample data before deployment.

This allows users to verify their parsing rules work correctly without deploying an adapter.

  • New USP.py module with validation logic
  • Calls LimaCharlie validation API (POST /usp/validate/{oid})
  • Built-in parser platforms (cef, wel, gcp, aws, 1password, duo, slack) don't require custom mapping
  • Multiple input methods: inline text, file, JSON array
  • Output formats: summary, json, yaml

Usage

# Text platform with custom mapping
limacharlie usp validate \
  --platform text \
  --mapping-file mapping.yaml \
  --input-file sample.log \
  --output-format json

# CEF platform (built-in parser, no mapping required)
limacharlie usp validate \
  --platform cef \
  --input-file cef-sample.log

# JSON platform with field mapping
limacharlie usp validate \
  --platform json \
  --mapping-file json-mapping.yaml \
  --input-file data.json \
  --json-input \
  --output-format json

Example Output

=== Testing Text Platform Parsing ===
Platform 'text' requires a custom mapping
VALIDATION SUCCESSFUL

Parsed 3 event(s):

[
  {
    "event_time": 0,
    "event_type": "INFO",
    "hostname": "server01",
    "index_records": null,
    "investigation_id": "",
    "json_payload": {
      "hostname": "server01",
      "level": "INFO",
      "message": "User login successful",
      "timestamp": "2024-01-01T12:00:00Z"
    },
    "sensor_key": "",
    "sid_ccs": null
  },
  {
    "event_time": 0,
    "event_type": "WARN",
    "hostname": "server01",
    "index_records": null,
    "investigation_id": "",
    "json_payload": {
      "hostname": "server01",
      "level": "WARN",
      "message": "High memory usage detected",
      "timestamp": "2024-01-01T12:00:05Z"
    },
    "sensor_key": "",
    "sid_ccs": null
  },
  {
    "event_time": 0,
    "event_type": "ERROR",
    "hostname": "server02",
    "index_records": null,
    "investigation_id": "",
    "json_payload": {
      "hostname": "server02",
      "level": "ERROR",
      "message": "Database connection failed",
      "timestamp": "2024-01-01T12:00:10Z"
    },
    "sensor_key": "",
    "sid_ccs": null
  }
]

Test plan

  • Run limacharlie usp validate --help to verify CLI registration
  • Test text platform with custom mapping file
  • Test JSON platform with --json-input flag
  • Test built-in parser platform (cef) without mapping
  • Verify error handling for missing required arguments
  • Verify non-zero exit code on validation failure

Links

Add new `limacharlie usp validate` command to test USP adapter parsing
configurations against sample data before deployment. Calls the
LimaCharlie validation API to verify parsing rules work correctly.

Features:
- Support for text, json, cef, gcp, aws platforms
- Built-in parser platforms (cef, wel, gcp, aws, 1password, duo, slack)
  don't require custom mapping configuration
- Multiple input methods: inline, file, JSON array
- Output formats: summary, json, yaml
- Support for multi-mapping selection via --mappings-file
- Optional indexing rules validation

Usage:
  limacharlie usp validate --platform text --mapping-file rules.yaml --input-file sample.log
  limacharlie usp validate --platform cef --input-file cef.log
  limacharlie usp validate --platform json --mapping-file map.yaml --input-file data.json --json-input
@tomaz-lc tomaz-lc force-pushed the feat/parse-validate branch from b179880 to 6cc4718 Compare January 29, 2026 17:58
@tomaz-lc tomaz-lc marked this pull request as ready for review January 30, 2026 08:42
Return error (exit code 1) when parsing produces no events in the
'limacharlie usp validate' command. Empty results typically indicate
misconfigured parsing rules or wrong platform type.

Add unit tests for empty results handling
@tomaz-lc tomaz-lc merged commit 0eff5c1 into master Jan 30, 2026
1 check passed
@tomaz-lc tomaz-lc deleted the feat/parse-validate branch January 30, 2026 17:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants