Skip to content

Code Challenge Submission#397

Open
ktross wants to merge 1 commit into
serpapi:masterfrom
ktross:master
Open

Code Challenge Submission#397
ktross wants to merge 1 commit into
serpapi:masterfrom
ktross:master

Conversation

@ktross

@ktross ktross commented Jul 17, 2026

Copy link
Copy Markdown

Summary

I implemented this solution using Codex, but I have committed relevant markdown docs and reviewed all code. It's a fairly close representation of the code that I would produce on a daily basis.

The initial solution I came up with was around 20-30ms to process the Van Gogh page, but I got that down to ~4ms. If I was more familiar with scraping Google or spent more time with it I could probably get it down further. I'm very happy with the result for a few hours of work.

image

If you include the ruby/system overhead it's about 130ms.

image

There are a few minor validation checks that I purposefully left out for performance reasons.

I initially designed this to support fallbacks when finding the artwork in the DOM, but [data-attrid="kc:/visual_art/visual_artist:works"] turned out to be the fastest and seemed the most stable based on my limited experience with Google search. If I were building something like this from scratch to actually use in production, I would have a list of markers sorted by ascending speed that I would try in order. I would log any failures (assuming LGTM stack) and set up anomaly detection to alert when the page shape changes.

CI is passing: https://github.com/ktross/code-challenge/actions/runs/29611810177

The remainder of this PR description is AI-generated from my requirements and docs.

Approach

  • Parse HTML with Nokolexbor.
  • Find Google’s visual-artist works section using its descriptive data-attrid.
  • Locate the card list by HTML structure rather than generated CSS classes.
  • Validate every card before returning any results.
  • Recover images from data-src, src, or saved _setImagesSrc assignments.
  • Resolve relative links against https://www.google.com without making requests.
  • Preserve card order and omit unavailable optional values.

Extraction is fully offline and does not execute JavaScript or fetch links and images.

macOS setup

The project uses Ruby 4.0.6 and Bundler 4.0.16 through Mise.

If Mise is already configured:

mise install
mise exec -- bundle install

For a new Mise installation:

brew install mise openssl@3 libyaml gmp rust
mise settings add idiomatic_version_file_enable_tools ruby
mise install
mise exec -- gem update --system 4.0.16
mise exec -- gem install bundler --version 4.0.16 --no-document
mise exec -- bundle install

Run the extractor

mise exec -- bin/extract files/van-gogh-paintings.html

The command prints formatted JSON to standard output.

Run the checks

mise exec -- bundle exec rake

Or separately:

mise exec -- bundle exec rspec
mise exec -- bundle exec rubocop

Test coverage

The suite contains 100 examples covering:

  • exact output for the supplied Van Gogh page;
  • exact output for four additional saved artist searches;
  • all 47 Van Gogh cards and their source order;
  • images stored in HTML attributes and page scripts;
  • blank optional fields and unavailable images;
  • malformed and ambiguous card structures;
  • unsafe or invalid links;
  • independence from Google-generated classes and JavaScript attributes;
  • extraction and benchmark command behavior.

A shared fixture inventory automatically includes every additional HTML and expected-JSON pair in the complete cross-fixture checks. Unpaired fixture files fail test loading.

Benchmark

mise exec -- bin/benchmark

This reports complete extraction time and focused diagnostic measurements. Results are intended for local comparisons rather than CI thresholds.

Design notes

  • Nokolexbor is the only runtime dependency.
  • Generated Google class names are deliberately not used as selectors.
  • A malformed card causes extraction to fail rather than return partial data.
  • The validated direct anchor and image are reused during field extraction.
  • Script-image parsing is intentionally limited to the formats observed in the saved pages.
  • Cards without a recoverable local image remain valid and omit the image field.

Verification

  • 100 RSpec examples passing
  • 21 files passing RuboCop
  • Exact reviewed JSON for all five saved pages
  • No network access during extraction or tests
  • Complete Van Gogh extraction measured locally at approximately 4.3 ms per run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant