Skip to content

FileSystemStorageClient writes snake_case data on disk instead of camelCase #1844

@janbuchar

Description

@janbuchar
  • @docs_group('Storage data')
    class RequestQueueMetadata(StorageMetadata):
    """Model for a request queue metadata."""
    model_config = ConfigDict(validate_by_name=True, validate_by_alias=True, from_attributes=True)
    had_multiple_clients: Annotated[bool, Field(alias='hadMultipleClients')]
    """Indicates whether the queue has been accessed by multiple clients (consumers)."""
    handled_request_count: Annotated[int, Field(alias='handledRequestCount')]
    """The number of requests that have been handled from the queue."""
    pending_request_count: Annotated[int, Field(alias='pendingRequestCount')]
    """The number of requests that are still pending in the queue."""
    total_request_count: Annotated[int, Field(alias='totalRequestCount')]
    """The total number of requests that have been added to the queue."""
    - models such as RequestQueueMetadata specify camelCase aliases
    • this suggests that we originally intended to maintain the same on-disk format as crawlee-js
    • FileSystemStorageClient calls model_dump without by_alias=True - that means that the files will be snake_case
  • I found this while working on the Rust file system storage client
  • @vdusek perhaps you remember if this was intentional?

Metadata

Metadata

Assignees

No one assigned

    Labels

    t-toolingIssues with this label are in the ownership of the tooling team.

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions