Formats and the Parser Factory

kymeer detects the format of each log line when it arrives, so one collector port can carry mixed traffic. Events are normalised to kymeer's canonical field schema. Logs that no parser recognises are kept, stay searchable, and become input for the Parser Factory.

Which formats are detected?

Format	Notes
Syslog	RFC 3164 and RFC 5424
JSON	A JSON object or array as the whole payload
CEF	Recognised inside a syslog envelope as well as on its own
LEEF	Recognised inside a syslog envelope as well as on its own
key=value	Lines made up mainly of key=value pairs

CEF and LEEF are checked first because they usually travel inside syslog.

What happens to fields that do not map?

They are kept. Fields that have no place in the canonical schema are stored with the event as unmapped fields, so nothing that was parsed is thrown away.

A single record larger than 1 MiB is not parsed. It is kept in the dead-letter store with the reason it was rejected.

How does the Parser Factory work?

1. It reads stored events that no parser recognised.
2. It works out the device type from the lines themselves. Devices it cannot identify wait in a confirmation queue.
3. It groups the lines by shape: lines with the same structure from the same device type. A shape is considered once it has been seen at least three times; a parser is never built from a single line.
4. It builds a candidate for the shape and checks it against the sample lines.
5. The shape goes live automatically only if all three checks pass: on every sample line at least 95% of the line's fields are mapped, it takes over no line that an existing parser already reads, and every sample agrees on the device type.
6. Otherwise the shape stays a draft, and a draft parses nothing. A shape with nothing to anchor it, such as free text with no key or message ID, stays a draft until a person adds a match rule.

A live shape applies to events as they arrive. Values a generated shape does not map stay on the event as unmapped fields.

If AI is set up, the Parser Factory can ask a model to help build a candidate. That call goes through the same masking and the same egress switch as every other AI call. With egress off, candidates are built without AI.

Are my logs used to build parsers for other customers?

The structure of your logs can be; your data is not.

kymeer keeps one shared parser for each device type, such as a FortiGate firewall, and every tenant with that device uses it. When the Parser Factory learns a new shape, it adds the shape to the shared parser for that device type, so every tenant with the device benefits.

• A published parser never contains a log line, or the identity of any tenant whose lines contributed. kymeer refuses to publish a parser that would.
• Values specific to a tenant are never used to match lines. Only allow-listed vendor values, such as a vendor's message IDs, are.
• Each promotion is recorded in the audit log of every tenant whose lines contributed, before the shape goes live. Every version of a shared parser is kept, so a change can be rolled back.

Who decides which parsers are used?

• Shared parsers, one per device type, are maintained by kymeer Platform Admins. The Parser Factory adds a shape to them only when it passes the checks above.
• Within your tenant, Senior Analysts and Tenant Admins can build parsers in the parser builder, for example for log lines the Parser Factory has not parsed. These apply only to your tenant. See Roles.

What about API connectors?

Some sources are collected through API connectors rather than agents. A connector is a declarative specification. The shared template holds no secrets; each tenant's copy stores its own credentials, encrypted, and has its own on/off switch. Your partner confirms which connectors fit your sources.