Code Challenge Submission#397
Open
ktross wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
I implemented this solution using Codex, but I have committed relevant markdown docs and reviewed all code. It's a fairly close representation of the code that I would produce on a daily basis.
The initial solution I came up with was around 20-30ms to process the Van Gogh page, but I got that down to ~4ms. If I was more familiar with scraping Google or spent more time with it I could probably get it down further. I'm very happy with the result for a few hours of work.
If you include the ruby/system overhead it's about 130ms.
There are a few minor validation checks that I purposefully left out for performance reasons.
I initially designed this to support fallbacks when finding the artwork in the DOM, but
[data-attrid="kc:/visual_art/visual_artist:works"]turned out to be the fastest and seemed the most stable based on my limited experience with Google search. If I were building something like this from scratch to actually use in production, I would have a list of markers sorted by ascending speed that I would try in order. I would log any failures (assuming LGTM stack) and set up anomaly detection to alert when the page shape changes.CI is passing: https://github.com/ktross/code-challenge/actions/runs/29611810177
The remainder of this PR description is AI-generated from my requirements and docs.
Approach
data-attrid.data-src,src, or saved_setImagesSrcassignments.https://www.google.comwithout making requests.Extraction is fully offline and does not execute JavaScript or fetch links and images.
macOS setup
The project uses Ruby 4.0.6 and Bundler 4.0.16 through Mise.
If Mise is already configured:
mise install mise exec -- bundle installFor a new Mise installation:
Run the extractor
mise exec -- bin/extract files/van-gogh-paintings.htmlThe command prints formatted JSON to standard output.
Run the checks
Or separately:
Test coverage
The suite contains 100 examples covering:
A shared fixture inventory automatically includes every additional HTML and expected-JSON pair in the complete cross-fixture checks. Unpaired fixture files fail test loading.
Benchmark
mise exec -- bin/benchmarkThis reports complete extraction time and focused diagnostic measurements. Results are intended for local comparisons rather than CI thresholds.
Design notes
imagefield.Verification