Skip to content

from_unix_timestamp should support float timestamps with subsecond precision #1478

@garysassano

Description

@garysassano

Summary

The from_unix_timestamp function in Vector's VRL only accepts integer values, which requires unnecessary and error-prone conversions when working with float Unix timestamps that include subsecond precision (e.g., from Caddy logs: 1692470616.525887).

Current Behavior

Currently, from_unix_timestamp has this signature:

from_unix_timestamp(value: <integer>, [unit: <string>]) :: <timestamp>

This means when you have a float timestamp like 1692470616.525887, you cannot use it directly and must perform manual conversion:

# Current workaround - verbose and error-prone
timestamp_ns = to_int(float!(.ts) * 1000000000)
.Timestamp = from_unix_timestamp!(timestamp_ns, unit: "nanoseconds")

Expected Behavior

The function should accept both integer and float values, automatically handling the conversion:

# Should work directly
.Timestamp = from_unix_timestamp!(.ts)  # Where .ts is 1692470616.525887

# Or with explicit unit specification
.Timestamp = from_unix_timestamp!(.ts, unit: "nanoseconds")

Proposed Solution

Update the function signature to:

from_unix_timestamp(value: <integer | float>, [unit: <string>]) :: <timestamp>

When a float is provided:

  • For unit: "seconds" (default): convert to DateTime, discarding the subsecond precision
  • For unit: "milliseconds": convert to DateTime64(6)
  • For unit: "nanoseconds": convert to DateTime64(9)

Environment

  • Vector version: 0.47.0
  • Platform: AL2023 on EC2

Metadata

Metadata

Assignees

No one assigned

    Labels

    type: bugA code related bugvrl: stdlibChanges to the standard library

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions