workaround for beatport

[karpuzikov]

We've had problems with the Beatport API for a while now.

This userscript can somehow access barcodes and ISRCs directly from Beatport release pages: https://github.com/murdos/musicbrainz-userscripts/blob/dist/beatport_importer.user.js

My idea is to create a way to import data directly from a release page into Harmony, perhaps using a cache, cookies, or another method.

The process could work like this:

1. Open the Beatport release page.
2. Load and extract the release data.
3. Import that data into Harmony.

This could serve as a fallback when the API isn't working. For example, Harmony could display: "The API is unavailable. Click here to import data manually." The Beatport page would open, the data would be copied, and the user could return to Harmony and paste or import it.

Maybe this is impossible to implement, I'm not a programmer but it looks like it could work.

[arsinclair]

Not saying supplying a loosely-structured data to Harmony is a bad idea, I personally can see some value in it. But the amount of restructuring it takes will probably overweight the benefits.

Building on that idea it is also technically possible to use the script running in the user browser session to scrape all the data needed by Harmony and then pass it during import. This could be done specifically for beatport, so a general loosely-structured metadata ingestion API is not necessary.

---

With that said, I just wanted to clarify a few things. If you wanna import a release with the data hosted specifically on Beatport, I suppose you could just use the Import functionality of that script?

We're currently not using Beatport API in Harmony because obtaining access to it is somewhat complicated. What was used before is a barcode lookup using Beatport's user-facing search. However, they had turned that off about a year ago, so barcode lookups broke.

If was still possible to import from Beatport and lookup on all other providers with a direct beatport release link, however roughly 6-7 months ago Beatport introduced bot protection using Cloudflare shields, which blocks requests from Harmony entirely, so this is what finally prevented all Beatport-related imports and lookups.

I suppose at a technical level the most viable solution to bypass the shields would be to copy the Cloudflare cookie with a solved challenge from Beatport to Harmony and make Harmony use it for the lookup requests (it's called cf_clearance). Alternatively, I think it is possible to host Harmony on your own and some users are doing it: this way you could for example put it behind Cloudflare WARP tunnel, which should help to bypass the bot protection.

[karpuzikov]

I suppose you could just use the Import functionality of that script?

Well I can use it to directly import from BP release page, but if click "import with harmony" it runs harmony search with the release url.
My idea was to combline data from the user session and srach by harmony for finding other sources

put it behind Cloudflare WARP tunnel, which should help to bypass the bot protection.

this is too much work)

[arsinclair]

Just thinking of it, forwarding CF cookie seems to be the easiest option...

I wrote to their support again asking for instructions how to obtain the API access.

[arsinclair]

It seems the API door has closed for good.

Email thread