Skip to content

Made some tweaks to the low-level code section to enable faster db lo…#1

Open
derekn97 wants to merge 1 commit into
Hugoberry:mainfrom
derekn97:main
Open

Made some tweaks to the low-level code section to enable faster db lo…#1
derekn97 wants to merge 1 commit into
Hugoberry:mainfrom
derekn97:main

Conversation

@derekn97

Copy link
Copy Markdown
  1. The previous Slice function constructed a new Binary Format.Record and effectively processed the preceding portion of the binary every time it extracted a range.
  • Update: Replaced it with Binary.Range to return a subset starting at an offset, so that it avoids repeatedly constructing the record-based parser.

  1. The code previously converted every one-, two-, and four-byte slice to a list and then accumulated its contents. This is especially expensive for varints because it happens multiple times per row.
  • Update: Defined reusable binary readers once (UInt16BEFormat, UInt32BEFormat, Float64BEFormat, ReadByte) to avoid creating a binary, converting it into a one-item list, and accumulating the list for every byte in every varint.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant