diff --git a/kits/bigquery-firestore-export/CHANGELOG.md b/kits/bigquery-firestore-export/CHANGELOG.md index 66f7e8b76..711eb60d3 100644 --- a/kits/bigquery-firestore-export/CHANGELOG.md +++ b/kits/bigquery-firestore-export/CHANGELOG.md @@ -1 +1 @@ -- Initial release +- Initial release of kit, see README for differences between the legacy extension and this kit diff --git a/kits/bigquery-firestore-export/README.md b/kits/bigquery-firestore-export/README.md index 884da7b8d..4fd7de069 100644 --- a/kits/bigquery-firestore-export/README.md +++ b/kits/bigquery-firestore-export/README.md @@ -173,6 +173,126 @@ The run document stores DTS metadata and row counts. Its `output` subcollection contains converted query rows. The `latest` document is updated transactionally so an older completion message cannot replace a newer run. +## Differences from the Export BigQuery to Firestore extension + +This kit is version 0.2.2 of the extension repackaged as an npm package. It is a +close port: the same two functions, the same BigQuery Data Transfer scheduled +query, the same `transferConfigs/{configId}/runs/{runId}` and `runs/latest` +documents, the same `WRITE_TRUNCATE` destination table naming and the same +row-by-row copy into Firestore. Every setting keeps its extension environment +variable name and default, so a `.env` copied from your installed instance needs +no value changes. What changes is the instance id, the Pub/Sub topic, the identity +the scheduled query runs as, and how repeated BigQuery columns land in Firestore. + +### You set `INSTANCE_ID` yourself, and the Pub/Sub topic is renamed + +The extension derived an instance id at install and used it to name its +notification topic (`ext--processMessages`) and to tag its transfer +config document with `extInstanceId`. Here `INSTANCE_ID` is a setting you +provide, and it must match this instance's key in the `instances` map in +`firebase.json`. + +The topic becomes `kit--processMessages`, and the kit creates it on +first run if it does not already exist. Set `INSTANCE_ID` to your installed +instance's id if you want the kit to adopt the scheduled query that instance +created, because the lookup is by `extInstanceId` on the documents in +`COLLECTION_PATH`. With a different id the kit finds nothing, creates a second +scheduled query, and you end up with two writing into the same collection. + +The existing transfer config still points its notifications at the old `ext-` +topic; the kit's update path rewrites `notification_pubsub_topic` to the new one +on the first deploy, so the old topic can be deleted afterwards. + +### Repeated BigQuery columns are now written as arrays + +A repeated (`ARRAY`) column used to arrive in Firestore as a map keyed by +position, `{ "0": ..., "1": ... }`, because the conversion treated every +non-scalar value as an object. The kit writes a real Firestore array instead. +Anything reading those fields by numeric string key needs updating, and rows +written before and after the change are not the same shape. Scalars, timestamps, +dates, times, datetimes, bytes and geography values convert exactly as before. + +### The scheduled query runs as a different service account + +The extension created the transfer config with +`serviceAccountName: ext-@.iam.gserviceaccount.com`, so the +query ran as the extension's own service account. The kit creates it without a +service account name, so BigQuery Data Transfer runs it as the identity that +created it, which is your function's runtime service account (the default compute +service account unless you have set one). + +That account needs to be able to read whatever `QUERY_STRING` touches and write +to `DATASET_ID`. `roles/bigquery.admin` on the function covers this for datasets +in the same project; a cross-project query needs the grant made explicitly. This +was not exercised against a live deploy. + +Note also that a service account cannot be changed on an existing transfer config, +so a scheduled query originally created by an installed extension instance keeps +running as the extension's service account even after the kit adopts it. Delete +and recreate the scheduled query if you want it moved. + +### The setup step runs on every deploy + +The install, update and configure hooks are replaced by an `upsertTransferConfig` +task that the CLI runs after your first deploy and after every redeploy. It does +the same work: create the scheduled query if this instance has none, otherwise +reconcile the existing one against your current `QUERY_STRING`, `DATASET_ID`, +`TABLE_NAME`, `PARTITIONING_FIELD` and `SCHEDULE`. + +Two consequences of it now being an ordinary task rather than a lifecycle event. +There is no install UI to report progress into, so failures show up in the task's +function logs, and the task retries up to five times with a 30 second minimum +backoff. And `DISPLAY_NAME` is no longer immutable, but changing it does not +rename an existing scheduled query, because display name is not part of the +update; it only applies to a config the kit creates. + +Removing `PARTITIONING_FIELD` once it has been set still fails, with the same +explanation, because the BigQuery Data Transfer API cannot clear it. + +### You can link an existing scheduled query + +`TRANSFER_CONFIG_NAME` is a new setting. Point it at the full resource name of a +scheduled query you already have +(`projects//locations//transferConfigs/`) and the kit +records that config in Firestore and consumes its notifications instead of +creating one of its own. The extension carried the code for this but no setting to +reach it. Leave it empty for the create-or-reconcile behaviour described above. + +### Region, and no location setting + +`LOCATION` is gone. Both functions deploy to your codebase's default region +(`us-central1` unless you have changed it) rather than the immutable location you +picked at install. `BIGQUERY_DATASET_LOCATION` is unchanged and still tells the +result query where your dataset lives. + +### Failed notifications are retried + +`processMessages` is a 2nd gen Pub/Sub function with retries enabled, where the +extension's 1st gen trigger did not retry. A run whose results fail to copy, for +example because BigQuery or Firestore is briefly unavailable, is now retried +rather than dropped. A notification that keeps failing, such as one for a transfer +config not tagged with this `INSTANCE_ID`, is also retried until Pub/Sub gives up. + +Both functions' service accounts need `roles/eventarc.eventReceiver` and +`roles/run.invoker` on top of the three roles the extension asked for, and the +Pub/Sub API is now requested explicitly. The Firebase CLI handles all of this. + +### Unchanged + +- `COLLECTION_PATH` still defaults to `transferConfigs`, and the document layout + under it is identical: the transfer config document keyed by config id, a `runs` + subcollection keyed by run id holding `runMetadata`, `totalRowCount` and + `failedRowCount`, a `latest` document, and an `output` collection of rows per + run. +- The destination table is still `TABLE_NAME_{run_time|"%H%M%S"}` with + `WRITE_TRUNCATE`, and results are still read with `SELECT *` in + `BIGQUERY_DATASET_LOCATION`. +- Runs that do not succeed still write a run document with zeroed counts and still + update `latest`, and `latest` is still only moved forward by a newer run. +- Rows are still written one document at a time in chunks of 10,000, with + per-row failures logged and counted rather than aborting the run. +- `LOG_LEVEL` still accepts `debug`, `info`, `warn`, `error` and `silent`. + ## API surface - **Main entry** (`@firebase/bigquery-firestore-export`): exports diff --git a/kits/delete-user-data/CHANGELOG.md b/kits/delete-user-data/CHANGELOG.md index 66f7e8b76..711eb60d3 100644 --- a/kits/delete-user-data/CHANGELOG.md +++ b/kits/delete-user-data/CHANGELOG.md @@ -1 +1 @@ -- Initial release +- Initial release of kit, see README for differences between the legacy extension and this kit diff --git a/kits/delete-user-data/README.md b/kits/delete-user-data/README.md index 5c7b81f20..3cc0ec9e3 100644 --- a/kits/delete-user-data/README.md +++ b/kits/delete-user-data/README.md @@ -135,6 +135,75 @@ When `EVENTARC_CHANNEL` is configured, the functions publish deletion events for each backend under `firebase.extensions.delete-user-data.v1.*` (`firestore`, `database`, and `storage`). +## Differences from the Delete User Data extension + +This kit is the extension repackaged as an npm package, but a few things behave +differently. If you are moving from an installed extension instance, read this +section before you deploy. + +### Auto-discovery uses `true` / `false` + +`ENABLE_AUTO_DISCOVERY` is a boolean param, and only the literal string `true` +enables it. The extension used `yes` / `no`, so copying an old config across +leaves auto-discovery silently switched off. Change `yes` to `true` in your +`.env`. + +### You set `INSTANCE_ID` yourself + +The extension derived an instance id at install time and used it to name the +Pub/Sub topics. Here it is a setting you provide, and it must match this +instance's key in the `instances` map in `firebase.json`. If the two disagree, +auto-discovery publishes to a topic nothing is listening on. + +### Pub/Sub topics are named differently + +Discovery and deletion topics are now `kit--discovery` and +`kit--deletion`, where the extension used an `ext-` prefix. The +Firebase CLI creates them for you on deploy, so there is no manual setup step, +but the old topics from an extension install are not reused and can be deleted +once you have migrated. + +You can also override both names with `DISCOVERY_TOPIC_NAME` and +`DELETION_TOPIC_NAME`, which the extension did not allow. Change them together, +since one function publishes to a topic the other is triggered by. + +### Realtime Database deletion no longer needs a database instance + +The extension only cleared RTDB paths when both `SELECTED_DATABASE_INSTANCE` +and `SELECTED_DATABASE_LOCATION` were set. This kit clears them whenever +`RTDB_PATHS` is set, falling back to your project's default database when no +instance is named. Set `SELECTED_DATABASE_INSTANCE` explicitly if you are +targeting a secondary database, and leave `RTDB_PATHS` empty if you do not want +RTDB touched at all. + +### Functions deploy to your default region + +The extension deployed to the location you picked at install time. This kit +sets no region, so its functions deploy to your codebase's default +(`us-central1` unless you have changed it). + +### Pub/Sub handlers are 2nd gen + +`handleSearch` and `handleDeletion` are now 2nd gen functions. `clearData` +stays 1st gen, because the Firebase Auth `user.delete` trigger has no 2nd gen +equivalent. This mainly matters if you have infrastructure or alerting keyed to +function generation. + +### Empty search fields no longer error + +Setting `AUTO_DISCOVERY_SEARCH_FIELDS` to an empty value used to raise an +invalid field path error during discovery. It is now treated as "match on the +document path only". The default is unchanged (`id,uid,userId`). + +### Unchanged + +Events are the same. When `EVENTARC_CHANNEL` is configured, the functions still +publish `firebase.extensions.delete-user-data.v1.firestore`, `.database` and +`.storage` with the same payloads. Path syntax (`{UID}` substitution, comma +separated lists, `{DEFAULT}` for the default Storage bucket), the shallow and +recursive Firestore delete modes, the search depth and field matching rules, +and the custom `SEARCH_FUNCTION` contract all behave as they did. + ## API surface - **Main entry** (`@firebase/delete-user-data`): exports `clearData`, diff --git a/kits/firestore-bigquery-export/CHANGELOG.md b/kits/firestore-bigquery-export/CHANGELOG.md index 66f7e8b76..711eb60d3 100644 --- a/kits/firestore-bigquery-export/CHANGELOG.md +++ b/kits/firestore-bigquery-export/CHANGELOG.md @@ -1 +1 @@ -- Initial release +- Initial release of kit, see README for differences between the legacy extension and this kit diff --git a/kits/firestore-bigquery-export/README.md b/kits/firestore-bigquery-export/README.md index cba06ef86..dd2f57fce 100644 --- a/kits/firestore-bigquery-export/README.md +++ b/kits/firestore-bigquery-export/README.md @@ -215,6 +215,75 @@ missing when a write arrives, the inline write fails, the handler calls failure is surfaced to the function runtime retry policy (`retry: true` on `fsexportbigquery`). +## Differences from the Stream Firestore to BigQuery extension + +This kit is the extension repackaged as an npm package, but a few things behave +differently. If you are moving from an installed extension instance, read this +section before you deploy. + +### Boolean settings use `true` / `false` + +`WILDCARD_IDS`, `USE_NEW_SNAPSHOT_QUERY_SYNTAX` and `EXCLUDE_OLD_DATA` are +boolean params, and only the literal string `true` enables them. The extension +used `yes` / `no` for the last two, so copying an old config across leaves them +silently disabled. Change any `yes` to `true` in your `.env`. + +### Failed writes retry differently + +The extension pushed a failed BigQuery write onto a Cloud Tasks queue +(`syncBigQuery`) and retried it from there. This kit has no task queue on the +write path. A failed write is retried once in place, and anything still failing +is handed to the Cloud Functions runtime retry policy, which redelivers the +Firestore event. + +The practical effects: retries no longer show up as a separate function or +queue in the console, and the two knobs that tuned that queue, +`MAX_DISPATCHES_PER_SECOND` and `MAX_ENQUEUE_ATTEMPTS`, no longer exist. + +### Events + +`onSuccess` is no longer published. The extension emitted it from the task +queue handler, which is gone, so the kit publishes `onStart` and `onError` +only. + +Events are published under `firebase.extensions.firestore-bigquery-export.v1.*` +only. The extension also published a duplicate copy of every event under +`firebase.extensions.firestore-counter.v1.*`, a historical naming mistake kept +for backwards compatibility. If you have Eventarc triggers listening on those +`firestore-counter` types, point them at the `firestore-bigquery-export` types. + +### Wildcard columns include the document ID + +With `WILDCARD_IDS=true`, the wildcard column now contains a `documentId` key +alongside the path parameters from your collection path. The extension wrote +the path parameters only. + +### Functions deploy to your Firestore region + +The extension let you pick a function location separately from the Firestore +database location. Here, `DATABASE_REGION` sets both: the trigger, the +lifecycle tasks, and the database being watched. + +### Defaults + +Two settings now have defaults rather than being passed through empty: +`DATASET_LOCATION` defaults to `us`, and `BIGQUERY_PROJECT_ID` defaults to the +project the functions are deployed to. + +### Tooling that is not included + +The extension shipped companion scripts that this package does not: + +- `fs-bq-import-collection`, for backfilling documents that already existed + before the export started. +- `gen-schema-view`, for generating strongly typed BigQuery views over the + changelog. +- The cross-project access grant scripts. + +`IMPORT_COLLECTION_PATH` is not a setting here. If you rely on any of these, +keep using the versions from the extension repository. They operate on the same +BigQuery changelog table, so they still work against data this kit writes. + ## API surface - **Main entry** (`@firebase/firestore-bigquery-export`): exports diff --git a/kits/firestore-bundle-builder/CHANGELOG.md b/kits/firestore-bundle-builder/CHANGELOG.md index 66f7e8b76..711eb60d3 100644 --- a/kits/firestore-bundle-builder/CHANGELOG.md +++ b/kits/firestore-bundle-builder/CHANGELOG.md @@ -1 +1 @@ -- Initial release +- Initial release of kit, see README for differences between the legacy extension and this kit diff --git a/kits/firestore-bundle-builder/README.md b/kits/firestore-bundle-builder/README.md index ebb7f7973..09358563c 100644 --- a/kits/firestore-bundle-builder/README.md +++ b/kits/firestore-bundle-builder/README.md @@ -119,6 +119,98 @@ Instance ids must be unique across all kit stanzas in the project, and every instance's function names are namespaced by its `kit--` prefix, so the instances cannot collide. +## Differences from the Firestore Bundle Builder extension + +This kit is the extension repackaged as an npm package. Config is a +lift-and-shift (`BUNDLESPEC_COLLECTION`, `BUNDLE_STORAGE_BUCKET` and +`STORAGE_PREFIX` keep their names, defaults and meanings), but several +behaviours changed. If you are moving from an installed extension instance, +read this section before you deploy. + +### Bundle specs are read per request + +The extension opened a snapshot listener on the whole spec collection at +startup and served every request from an in-memory copy. This kit reads the +spec document directly on each request. + +Three consequences: + +- Spec edits take effect immediately, with no dependence on listener delivery, + and there is no cold-start window where a request waits for the first + snapshot. +- A deleted spec now returns 404. The extension kept serving it, because + entries were only ever added to the in-memory map, never removed. +- Each request costs one document read. If you serve high volumes of + uncacheable bundles, budget for that. + +### The Cloud Storage cache behaves differently on a miss + +When a spec sets `fileCache`, the extension asked Cloud Storage for a read +stream without first checking the object existed. A missing object failed +asynchronously, after the response was already being written. This kit confirms +the object exists before streaming, and falls through to rebuilding the bundle +when it does not. + +Failures writing the built bundle back to Cloud Storage are now logged rather +than left unhandled. The response is still served from the freshly built +bundle. + +### `fileCache` is not a time-to-live + +Worth stating plainly, since the name suggests otherwise: a cached bundle is +served regardless of age. `fileCache` controls *whether* a bundle is cached, +not for how long. This matches the extension, which accepted a `ttlSec` value +and never enforced it. Use `clientCache` and `serverCache` for cache-control +headers if you need expiry. + +### Path parameters are validated + +Parameter values substituted into a bundle spec's document or collection path +are now rejected if they contain a `/`, or if they resolve to an empty value. +Both cases return an invalid-argument error and are logged. The extension +substituted them as-is, which allowed a caller to reach a path the spec author +did not intend. + +If a spec legitimately relies on a parameter expanding to a multi-segment path, +it will now fail. Split it into separate parameters, one per path segment. + +### Requests without a bundle ID return 404 + +A request to the function root, or with a trailing slash, returns 404 with the +usual "could not find bundle" message. The extension returned a 500 in that +case. Note the ID is the last path segment, not a query parameter, so `?id=x` +has never selected a bundle. + +### Comma-separated values for `in` queries + +A query condition using `in` or `not-in` accepts a comma-separated string and +splits it. Non-string values are now passed through unchanged rather than +being forced through string splitting, which used to throw. + +### Region + +`serve` deploys to `us-central1`, the same region the extension pinned. This is +fixed by the package rather than chosen at install time. + +### Disabling the Storage cache + +Setting `BUNDLE_STORAGE_BUCKET` to an empty value disables the Storage cache +outright, and specs with `fileCache` are built fresh on every request. The +default is your project's default Storage bucket, as before. + +### The admin dashboard is not included + +The extension shipped a separate Remix admin dashboard for authoring bundle +specs. It is not part of this package. Bundle spec documents are ordinary +Firestore documents, so you can keep using the dashboard from the extension +repository against the same collection, or write the documents yourself. + +### Runtime + +The functions run on Node 22 with 2nd gen Cloud Functions, where the extension +was on Node 14 with 1st gen. Bundle format and the client-side APIs for loading +bundles are unaffected. + ## API surface - **Main entry** (`@firebase/firestore-bundle-builder`): exports `serve`. The diff --git a/kits/firestore-counter/CHANGELOG.md b/kits/firestore-counter/CHANGELOG.md index 66f7e8b76..711eb60d3 100644 --- a/kits/firestore-counter/CHANGELOG.md +++ b/kits/firestore-counter/CHANGELOG.md @@ -1 +1 @@ -- Initial release +- Initial release of kit, see README for differences between the legacy extension and this kit diff --git a/kits/firestore-counter/README.md b/kits/firestore-counter/README.md index f62189969..2e80bbf67 100644 --- a/kits/firestore-counter/README.md +++ b/kits/firestore-counter/README.md @@ -114,6 +114,96 @@ When `EVENTARC_CHANNEL` is configured, the functions publish lifecycle events such as `onStart`, `onError`, `onSuccess`, and `onCompletion` under `firebase.extensions.firestore-counter.v1.*`. +## Differences from the Distributed Counter extension + +This kit is the extension repackaged as an npm package. It is a very close port: +both settings keep their name, type and default, so an existing `.env` is a +lift-and-shift, and the counting itself is untouched. Shards still live in the +`_counter_shards_` subcollection, are aggregated by the same algorithm on the +same schedule, and your existing security rules and client code keep working +without changes. The differences below are worth knowing before you deploy. + +### Your settings are no longer validated before deploy + +`INTERNAL_STATE_PATH` must be a document path, meaning an even number of +segments such as `_firebase_ext_/sharded_counter`. The extension rejected +anything else at install time. Nothing checks it now, so a collection path such +as `_firebase_ext_` deploys happily and then throws on every controller run: + +``` +Value for argument "documentPath" must point to a document, but was +"_firebase_ext_". Your path does not contain an even number of components. +``` + +Counters silently stop aggregating, and the only sign is the error in your +function logs. + +`SCHEDULE_FREQUENCY` is still a plain number of minutes, and the kit builds the +same `every N minutes` schedule from it. It is also unvalidated now, so a value +like `*/5 * * * *` or `5 minutes`, which the install prompt used to reject, +reaches your deploy instead. + +### The worker function publishes no events + +The extension published `onError` from all three of its functions. In the kit, +the worker function does not: the Eventarc channel is only configured on the +controller and shard-write functions. Failures that happen while a worker is +aggregating, which is the path that handles counters big enough to need workers, +now show up only in the logs. Events from the controller and shard-write +functions are unaffected. + +### Events must be wired up by hand + +Enabling events was part of the extension's install flow, which created the +channel and set the environment for you. The kit reads `EVENTARC_CHANNEL` and +`EXT_SELECTED_EVENTS` straight from the environment and the CLI never prompts +for them, so no events are published until you create a channel and put both +values in your `.env`. If you set `EVENTARC_CHANNEL` and leave +`EXT_SELECTED_EVENTS` unset, every event type is published. + +### Event payloads have a different shape + +The event types are unchanged, but what they carry is not. `onStart` used to +carry `{change, context}` and now carries `{data, params}`: the write is under +`data` instead of `change`, and the 1st gen `context` is gone. `onCompletion` +used to carry `{context}` and now carries `{params}` only. Anything reading +`context.eventId`, `context.timestamp`, `context.eventType` or +`context.resource` from these events needs updating; the trigger wildcards +(`collection`, `counter`, `shardId`) survive as `params`. + +### Your codebase's global options apply to these functions + +The functions are exported from your own functions codebase, so a +`setGlobalOptions` call there applies to them: region, memory, and instance +limits. The extension deployed with fixed settings you could not influence, and +always in `us-central1`. + +The controller and shard-write functions keep their own limit of one instance, +which a global setting does not override, so the single-writer behaviour is +safe. The worker function has no limit of its own and does pick up a global +`maxInstances`. The extension left workers unbounded, so a low global cap now +throttles aggregation exactly when the controller wants to spread the work over +many workers. + +### Client samples and the stress test app are not in the package + +The extension repo shipped counter clients for Web, Node, Android, iOS and Dart +plus a stress test app. The npm package contains only the functions. Nothing +about the shard layout changed, so the clients you already use keep working; +carry on getting them from the extension repo. + +### Unchanged + +- Both settings, with the same names and the same defaults + (`_firebase_ext_/sharded_counter`, `1` minute). +- The `_counter_shards_` subcollection name, the shard document format, and + therefore your security rules. +- The aggregation behaviour: inline aggregation up to 200 shards, workers above + that, 45 second self-scheduling worker runs, partial shard cleanup, and + deletion of shards once they are summed into the counter field. +- The three functions and the event types they publish, aside from the worker + and payload points above. + ## API surface - **Main entry** (`@firebase/firestore-counter`): exports `controllerCore`, diff --git a/kits/firestore-genai-chatbot/CHANGELOG.md b/kits/firestore-genai-chatbot/CHANGELOG.md index 66f7e8b76..711eb60d3 100644 --- a/kits/firestore-genai-chatbot/CHANGELOG.md +++ b/kits/firestore-genai-chatbot/CHANGELOG.md @@ -1 +1 @@ -- Initial release +- Initial release of kit, see README for differences between the legacy extension and this kit diff --git a/kits/firestore-genai-chatbot/README.md b/kits/firestore-genai-chatbot/README.md index fd9922188..d4b09cd30 100644 --- a/kits/firestore-genai-chatbot/README.md +++ b/kits/firestore-genai-chatbot/README.md @@ -134,6 +134,91 @@ Instance ids must be unique across all kit stanzas in the project, and every instance's function names are namespaced by its `kit--` prefix, so the instances cannot collide. +## Differences from the Build Chatbot with the Gemini API extension + +This kit is version 0.0.19 of the extension repackaged as an npm package you add +to your own functions codebase. The generation logic, the Firestore trigger, the +`status` state machine, the per-discussion overrides and the safety settings are +all ported verbatim. Config keeps the same environment variable names, so a +`.env` copied from your installed instance is close to a lift-and-shift, with +four exceptions below: the boolean toggles, the two region settings, and the API +key secret. + +### Change `yes` and `no` to `true` and `false` + +`ENABLE_DISCUSSION_OPTION_OVERRIDES` and `ENABLE_GENKIT_MONITORING` were +`yes`/`no` dropdowns. They are now booleans that count as enabled only for the +exact value `true`. A copied `.env` carrying `yes` deploys without complaint and +silently leaves the feature off, so per-discussion overrides stop being read and +Genkit monitoring stops reporting. + +### Pick your Cloud Functions region, or you get us-central1 + +The extension's `LOCATION` setting is gone. There is no replacement value, and +`LOCATION` left in a `.env` file is ignored. The function deploys to the Cloud +Functions default region, `us-central1`, wherever your extension instance used +to run. If you need another region, register the trigger yourself from the +package's `./lib` entry point and set `region` on it. + +### Set `VERTEX_AI_MODEL_LOCATION` explicitly if you use Vertex AI + +The default value `null` used to mean "call Vertex AI in the same region as the +function". It now means "let the SDK choose": `us-central1` for a normal +single-candidate request, and `global` when `CANDIDATE_COUNT` is above 1. If you +relied on the default to keep model calls in your function's region, set the +region by name instead of leaving it at `null`. + +### An API_KEY secret is required even on Vertex AI + +`API_KEY` was optional, so a Vertex AI instance could be installed without one. +It is now always bound to the function. If no `API_KEY` secret exists in Secret +Manager, `firebase deploy` prompts you for a value, and fails outright when +running non-interactively (CI). Create the secret with any placeholder value if +your provider is `vertex-ai`. + +### Long generations now time out after 60 seconds + +The extension ran with a 540 second timeout. The kit does not set one, so the +platform default of 60 seconds applies. Prompts with a long history or a high +`MAX_OUTPUT_TOKENS` that used to finish will now fail and write `status.state: +ERROR`. There is no config value for this; raise it on your own trigger from +`./lib` if you need the old headroom. + +### CANDIDATE_COUNT above 1 now really requests that many candidates + +With `CANDIDATE_COUNT` above 1, the extension never forwarded the count (nor +`TEMPERATURE`, `TOP_P` or `TOP_K`) to the model, so it wrote a `candidates` array +holding the single response it got back. The kit forwards all four, so you get +the number of candidates you asked for, your sampling settings take effect, and +the request costs more. Two smaller consequences: with per-discussion overrides +enabled, a `candidateCount` set on a discussion document now decides whether the +`candidates` field is written for that message (the extension decided once, from +the deploy-time value), and a discussion that overrides `candidateCount` above 1 +gets a `candidates` field containing one entry. + +### Bad numbers are no longer rejected up front + +`TEMPERATURE`, `TOP_P`, `TOP_K`, `CANDIDATE_COUNT`, `MAX_OUTPUT_TOKENS` and +`COLLECTION_NAME` were validated when you installed the extension. Nothing +validates them now: a non-numeric value is parsed to `NaN` and passed to the +model call rather than being caught at deploy time. + +### Unchanged + +- The watched path is still `COLLECTION_NAME/{messageId}` on the default + database, and `COLLECTION_NAME` still defaults to `generate`. +- `PROMPT_FIELD`, `RESPONSE_FIELD`, `ORDER_FIELD` and `CANDIDATES_FIELD` behave + identically, as does history assembly from sibling documents. +- The `status` state machine is unchanged, including that a document + already in `COMPLETED` or `ERROR` is never reprocessed when you edit its + prompt. +- Per-discussion overrides, `examples` and `continue` history are parsed and + validated exactly as before. +- The four `HARM_CATEGORY_*` thresholds, `CONTEXT` handling (still sent as a + leading system turn) and the user-facing error messages are unchanged. +- `MODEL` still defaults to `gemini-2.5-flash`, and the same list of supported + Gemini models is accepted. + ## API surface - **Main entry** (`@firebase/firestore-genai-chatbot`): exports diff --git a/kits/firestore-send-email/CHANGELOG.md b/kits/firestore-send-email/CHANGELOG.md index 66f7e8b76..711eb60d3 100644 --- a/kits/firestore-send-email/CHANGELOG.md +++ b/kits/firestore-send-email/CHANGELOG.md @@ -1 +1 @@ -- Initial release +- Initial release of kit, see README for differences between the legacy extension and this kit diff --git a/kits/firestore-send-email/README.md b/kits/firestore-send-email/README.md index bd17b7806..16a6d7719 100644 --- a/kits/firestore-send-email/README.md +++ b/kits/firestore-send-email/README.md @@ -134,6 +134,101 @@ such as `onStart`, `onProcessing`, `onSuccess`, `onError`, `onComplete`, `onPending`, and `onRetry` under `firebase.extensions.firestore-send-email.v1.*`. +## Differences from the Trigger Email from Firestore extension + +This kit is version 0.2.10 of the extension repackaged as an npm package you add +to your own functions codebase. Delivery, the `delivery` state machine, the +lease and retry handling, Handlebars templates and partials, UID recipient +lookup, the SendGrid transport, payload validation and the TTL field are all +ported verbatim. Every setting keeps its extension environment variable name and +default, so a `.env` copied from your installed instance needs no value changes. +What does change is where the four secrets come from, where the function runs, +and what is no longer checked or set up for you. + +### Create the four secrets by name, all of them + +The extension stored each secret param as `ext--` in Secret +Manager. The kit asks for secrets named exactly `SMTP_PASSWORD`, `CLIENT_ID`, +`CLIENT_SECRET` and `REFRESH_TOKEN`, so your existing extension secrets are not +picked up. All four are attached to the function whatever `AUTH_TYPE` is set to, +and were optional in the extension. If a secret does not exist, `firebase deploy` +prompts you for a value, and fails outright when running non-interactively (CI). +On username/password auth create the three OAuth2 secrets with a placeholder +value, and on OAuth2 auth do the same for `SMTP_PASSWORD`. + +### DATABASE_REGION now decides where the function runs + +In the extension it only told the trigger where your database lived; the function +itself ran in the Cloud Functions location you picked at install. The kit passes +`DATABASE_REGION` straight through as the function's region, so the function +moves to your database's region and the install-time location setting has no +replacement. If your Firestore is multi-region or dual-region (`nam5`, `nam7`, +`eur3`), that value is not a Cloud Functions region and the deploy fails; deploy +the trigger yourself from the package's `./lib` entry point with a real region +such as `us-central1` or `europe-west1`. This was not exercised against a live +deploy. + +### Create the Eventarc channel yourself for events + +Choosing events at install used to create the channel and set both event +variables for you. The kit only reads them: set `EVENTARC_CHANNEL` in your `.env` +to a channel you have created, and the same seven +`firebase.extensions.firestore-send-email.v1.*` events are published. Per-event +selection is gone in practice, because the CLI rejects any `.env` key beginning +with `EXT_`, so +`EXT_SELECTED_EVENTS` cannot be set and every event type is published. With +`EVENTARC_CHANNEL` unset, nothing is published and the function is otherwise +unaffected. + +### Nothing checks your settings at deploy time + +The extension rejected a malformed `DEFAULT_FROM`, a `MAIL_COLLECTION` that was +not a valid collection path, an `SMTP_CONNECTION_URI` that was not +`smtp(s)://...:port`, and a `TTL_EXPIRE_VALUE` that was not a positive integer, +before it would install. None of that is checked now. A bad from address or +connection URI deploys cleanly and every document fails at send time with +`delivery.state: ERROR` instead. `TTL_EXPIRE_VALUE: 0` is silently treated as +`1`, and a negative value produces a `delivery.expireAt` in the past, which a TTL +policy will act on immediately. + +### The setup steps and the OAuth2 helper are not in the README + +Two install-time instructions have no equivalent here. Automatic deletion still +needs you to create a Firestore TTL policy on `delivery.expireAt` by hand for the +collection the function watches, and the SendGrid guidance (categories, dynamic +templates, the `sendgridQueueId` in `delivery.info`) is documented only in the +extension. Both still apply unchanged. The standalone +`oauth2-refresh-token-helper.js` script is not shipped with the package, but it +is a plain download from the extension repository and still works for generating +a refresh token. + +### A missing template name now says so + +Rendering a template whose name does not exist in your templates collection wrote +a `TypeError` about reading `attachments` into `delivery.error`. It now writes +`Tried to render non-existent template ''`. + +### Unchanged + +- The watched path is still `MAIL_COLLECTION/{documentId}`, still matched as a + path pattern so nested collections such as `users/{uid}/mail` keep working, and + `MAIL_COLLECTION` still defaults to `mail`. +- Every environment variable keeps its name, type and default, including + `OAUTH_SECURE`, which was a `true`/`false` dropdown and is now a boolean that + reads those same two values. +- Document fields and their meanings are identical: `to`, `cc`, `bcc`, the + `*Uids` variants, `message`, `template`, `sendGrid`, `headers`, `categories`, + `from` and `replyTo`, along with the validation error messages written to + `delivery.error`. +- The `delivery` state machine is unchanged, including the 60 second processing + lease, that a document in `SUCCESS` or `ERROR` is never reprocessed, and the + `delivery.info` shape. +- SendGrid is still selected by an `smtp.sendgrid.net` connection URI with the + API key in `SMTP_PASSWORD`, and Outlook hosts still get their explicit + transport configuration. +- The function still runs with a 120 second timeout, and `TLS_OPTIONS`, + `DATABASE`, `USERS_COLLECTION` and `TEMPLATES_COLLECTION` behave as before. + ## API surface - **Main entry** (`@firebase/firestore-send-email`): exports `processQueue`. The diff --git a/kits/firestore-translate-text/CHANGELOG.md b/kits/firestore-translate-text/CHANGELOG.md index 66f7e8b76..711eb60d3 100644 --- a/kits/firestore-translate-text/CHANGELOG.md +++ b/kits/firestore-translate-text/CHANGELOG.md @@ -1 +1 @@ -- Initial release +- Initial release of kit, see README for differences between the legacy extension and this kit diff --git a/kits/firestore-translate-text/README.md b/kits/firestore-translate-text/README.md index 2d2281ca2..f7d190c1b 100644 --- a/kits/firestore-translate-text/README.md +++ b/kits/firestore-translate-text/README.md @@ -123,6 +123,107 @@ When `EVENTARC_CHANNEL` is configured, the function publishes lifecycle events such as `onStart`, `onError`, `onSuccess`, and `onCompletion` under `firebase.extensions.firestore-translate-text.v1.*`. +## Differences from the Translate Text in Firestore extension + +This kit is version 0.1.30 of the extension repackaged as an npm package. The +translation behaviour is ported closely: the same write trigger, the same +handling of string and map inputs, the same per-document `languages` override, +the same "delete the translations when the input goes away" rule, and the same +output shape written back to the document. Every setting keeps its extension +environment variable name and default, so a `.env` copied from your installed +instance needs no value changes. What changes is where the Gemini API key comes +from, which region Vertex AI is called in, and what is no longer checked for you. + +### Create a `GOOGLE_AI_API_KEY` secret even if you do not use Google AI + +The extension stored this as `ext--GOOGLE_AI_API_KEY` in Secret +Manager and it was optional. The kit asks for a secret named exactly +`GOOGLE_AI_API_KEY`, so your existing extension secret is not picked up, and the +secret is attached to the function whatever `TRANSLATION_PROVIDER` is set to. If +it does not exist, `firebase deploy` prompts you for a value and fails outright +when running non-interactively (CI). On the Cloud Translation or Vertex AI +providers, create it with a placeholder value. + +The key is only read when `TRANSLATION_PROVIDER` is `gemini-googleai`, and that +provider still fails fast with `Google AI API key is required for Genkit Google +AI translations` when the value is empty. + +### Vertex AI is called in the function's region + +With `TRANSLATION_PROVIDER: gemini-vertexai`, the Vertex AI call now uses the +region the function is deployed to. The extension used the location you picked at +install time. Gemini is not served in every region, so if you deploy somewhere it +is unavailable, translation fails and the error is written to your function logs. +Deploy to a region with Vertex AI support, or use `gemini-googleai` or +`translate` instead. This was not exercised against a live deploy. + +The function itself has no location setting any more. It deploys to your +codebase's default region (`us-central1` unless you have changed it). + +### Nothing checks your settings at deploy time + +The extension rejected a `LANGUAGES` value that was not a comma-separated list of +language codes, and a `COLLECTION_PATH` that was not a valid collection path, +before it would install. Neither is checked now, and `INPUT_FIELD_NAME`, +`OUTPUT_FIELD_NAME` and `LANGUAGES` all have defaults rather than being required, +so an incomplete or malformed config deploys and then fails per document at +translation time. `TRANSLATION_PROVIDER` falls back to `translate` when it is +empty. + +The two checks that ran per document still run: the function refuses to translate +when the input and output field names are the same, or when the input field name +is itself a path inside the output field. + +### Create the Eventarc channel yourself for events + +Choosing events at install used to create the channel and set both event +variables for you. The kit only reads them: set `EVENTARC_CHANNEL` in your `.env` +to a channel you have created, and the same four +`firebase.extensions.firestore-translate-text.v1.*` events are published. +Per-event selection is gone in practice, because the CLI rejects any `.env` key +beginning with `EXT_`, so `EXT_SELECTED_EVENTS` cannot be set and every event +type is published. With `EVENTARC_CHANNEL` unset, nothing is published and the +function is otherwise unaffected. + +### Event payloads have a different shape + +The event types are unchanged, but what `onStart` and `onCompletion` carry is +not. `onStart` used to carry `{change, context}` and now carries `{data, params}`: +the write is under `data` instead of `change`, and the 1st gen `context` is gone. +`onCompletion` used to carry `{context}` and now carries `{params}` only. Anything +reading `context.eventId`, `context.timestamp`, `context.eventType` or +`context.resource` needs updating; the `messageId` trigger wildcard survives as +`params`. `onSuccess` and `onError` payloads are unchanged. + +### No backfill + +There is no function to translate documents that already exist in the +collection. The extension carried the same limitation (its backfill task queue +was disabled and never deployed), so this is not a regression, but the code is +gone rather than dormant: only documents written after you deploy are translated. + +### The trigger is 2nd gen + +`fstranslate` is a 2nd gen Firestore function where the extension was 1st gen. +Its service account needs `roles/eventarc.eventReceiver` and `roles/run.invoker` +on top of `roles/datastore.user`; the Firebase CLI grants these for you. The +Cloud Translation API is still required whichever provider you choose. + +### Unchanged + +- The watched path is still `COLLECTION_PATH/{messageId}` on your default + database, and `COLLECTION_PATH` still defaults to `translations`. +- Every environment variable keeps its name, type and default, including the + three `TRANSLATION_PROVIDER` values and the three `GEMINI_MODEL` values. +- Duplicate entries in `LANGUAGES` are still collapsed. +- A string input is translated into every language; a map input has each of its + string values translated, with non-string values written as `null`. +- The per-document `LANGUAGES_FIELD_NAME` override, and the rule that a document + is re-translated only when its input or its languages actually change, both + behave as before. +- Translations are still written in a transaction, and each `onSuccess` event + still carries the output field name and the translations. + ## API surface - **Main entry** (`@firebase/firestore-translate-text`): exports `fstranslate`. diff --git a/kits/firestore-vector-search/CHANGELOG.md b/kits/firestore-vector-search/CHANGELOG.md index 66f7e8b76..711eb60d3 100644 --- a/kits/firestore-vector-search/CHANGELOG.md +++ b/kits/firestore-vector-search/CHANGELOG.md @@ -1 +1 @@ -- Initial release +- Initial release of kit, see README for differences between the legacy extension and this kit diff --git a/kits/firestore-vector-search/README.md b/kits/firestore-vector-search/README.md index e50d98f3e..e9cefbbd3 100644 --- a/kits/firestore-vector-search/README.md +++ b/kits/firestore-vector-search/README.md @@ -150,6 +150,176 @@ When `EVENTARC_CHANNEL` is configured, the functions publish lifecycle events such as `onStart`, `onError`, `onSuccess`, and `onCompletion` under `firebase.extensions.firestore-vector-search.v1.*`. +## Differences from the Vector Search with Firestore extension + +This kit is version 0.1.3 of the extension repackaged as an npm package, and it is +the least literal of the ports. The seven functions, the Firestore vector index, +the query document collection and the callable all survive with their names and +settings intact, so a `.env` copied from your installed instance needs no value +changes. The embedding providers, the backfill, and the shape of the status field +written onto your documents all changed, so read this before you point the kit at +a collection an installed instance has already embedded. + +### `EMBEDDING_PROVIDER: multimodal` is not implemented + +Selecting `multimodal` deploys, and then every embedding attempt throws +`Multimodal embeddings are not implemented in this package`. The extension's +multimodal image embedding, including reading images out of Cloud Storage, has no +equivalent here. If you use it, stay on the extension. + +### OpenAI embeddings are a different model and a different size + +`EMBEDDING_PROVIDER: openai` used `text-embedding-ada-002` and stored the full +1536-dimension vector, while the Firestore index it created was declared with 512 +dimensions. The kit uses `text-embedding-3-small` at 512 dimensions, which matches +the index. + +Vectors from the two models are not comparable, and the existing index is reused +as-is because the "does this index already exist" check only looks at the field +path, not the dimension. Re-embed the whole collection after you switch, and +delete the old vector index first if it was created with a different dimension. + +### You set `INSTANCE_ID` yourself, and it names the query collection + +The extension derived its instance id at install and used it for the query +collection (`_/index/queries`), the index metadata document +(`_/index`) and its task queues. Here `INSTANCE_ID` is a setting you +provide, and it must match this instance's key in the `instances` map in +`firebase.json`. To keep serving the query documents your clients already write +to, set it to your installed instance's id. The four task queue names can also be +overridden individually with `UPDATE_TRIGGER_QUEUE_NAME`, `UPDATE_TASK_QUEUE_NAME`, +`BACKFILL_TRIGGER_QUEUE_NAME` and `BACKFILL_TASK_QUEUE_NAME`, which the extension +did not allow. + +### Create the `GEMINI_API_KEY` and `OPENAI_API_KEY` secrets, both of them + +The extension stored these as `ext--GEMINI_API_KEY` and +`ext--OPENAI_API_KEY`, and both were optional. The kit asks for +secrets named exactly `GEMINI_API_KEY` and `OPENAI_API_KEY`, so your existing +extension secrets are not picked up, and both are attached to every function +whatever `EMBEDDING_PROVIDER` is set to. If either does not exist, `firebase +deploy` prompts you for a value and fails outright when running +non-interactively (CI). Create the one you do not need with a placeholder value. + +### `UPDATE_ON_CONFIGURE` now re-embeds on every deploy + +This setting was declared by the extension but never read. Reconfiguring an +installed instance re-embedded documents only when the provider, the vector +dimension or the input/output field names had actually changed, which the +extension tracked in its index metadata document. + +The kit keeps no such metadata and does no comparison. `UPDATE_ON_CONFIGURE: true` +enqueues a full re-embed of every document that already has an embedding after +*every* `firebase deploy`, whether anything relevant changed or not, and +`DO_BACKFILL: true` embeds the whole collection after the first deploy. On a large +collection that is a large Vertex AI or OpenAI bill per deploy. Set +`UPDATE_ON_CONFIGURE: false` and re-embed deliberately when you change providers. + +### Backfill is one task per document, and reads the collection in one go + +The extension chunked the collection into batches sized to the provider (16 +documents per OpenAI call), embedded each batch in a single API call, and tracked +progress in its metadata document. The kit reads the entire collection with one +`get()` and enqueues one Cloud Task per document, each of which embeds one +document with one API call. + +Two consequences. A collection large enough that a single `get()` does not fit in +the trigger's 512 MiB will fail the backfill outright, and there is no +resume-from-progress. Backfilling *n* documents now costs *n* task invocations and +*n* embedding calls rather than *n*/batch size. + +There is also no install-time progress reporting, since there is no extension +install UI to report into. Watch the function logs instead. + +### The `status` field on your documents is a different shape + +The extension wrote status nested under the process id, with timestamps: + +``` +status: { : { state: "COMPLETED", startTime, updateTime, completeTime, createTime } } +``` + +The kit writes it flat, with no timestamps: + +``` +status: { state: "COMPLETED" } +status: { state: "ERROR", message: "" } +``` + +The states themselves are narrower too: `PROCESSING` and `BACKFILLED` are no +longer written, only `COMPLETED` and `ERROR`. Anything reading +`status..state`, or a security rule or index keyed to it, needs +updating. The field name is still `STATUS_FIELD_NAME`, defaulting to `status`. + +Query documents no longer get a status field at all. They previously carried +`status.textQuery`, so if you were waiting on that to know a query had finished, +wait for `result` instead. + +### Editing a document's input re-embeds it + +The extension embedded each document once. Its skip rule was "this document's +status is already in a final state", so once a document reached `COMPLETED` (or +`ERROR`), changing its input field never produced a new embedding and a failure +was never retried. + +The kit compares the input instead: it re-embeds when the input field changes, and +skips only when the input is unchanged and an embedding is already present. This +is usually what you wanted, but it means editing inputs in bulk now costs +embedding calls, and a document that previously sat stale will be brought up to +date on its next write. + +### The lifecycle hooks and the function region + +Install and reconfigure hooks are replaced by an `initVectorSearch` task that the +CLI runs after your first deploy and after every redeploy. It creates the +Firestore vector index (skipping creation when a matching index exists, as +before) and then enqueues the backfill or update triggers according to the two +settings above. + +`LOCATION` is gone. The functions deploy to your codebase's default region +(`us-central1` unless you have changed it), and with +`EMBEDDING_PROVIDER: vertex` the Vertex AI embedding call uses that same region +rather than the install-time location. Gemini embedding is not served in every +region; if you deploy somewhere it is unavailable, embedding fails and the error +is written to the document's status field. + +### Events are actually published now + +The extension declared four event types but never published any. The kit +publishes `onStart`, `onSuccess`, `onError` and `onCompletion` under +`firebase.extensions.firestore-vector-search.v1.*` from `embedOnWrite`, once you +set `EVENTARC_CHANNEL` in your `.env` to a channel you have created. Per-event +selection is not available, because the CLI rejects any `.env` key beginning with +`EXT_`, so `EXT_SELECTED_EVENTS` cannot be set and every event type is published. +With `EVENTARC_CHANNEL` unset, nothing is published. + +### The triggers are 2nd gen + +All seven functions are 2nd gen. Their service accounts need +`roles/eventarc.eventReceiver`, `roles/run.invoker`, `roles/cloudtasks.enqueuer` +and `roles/iam.serviceAccountUser` on top of the four roles the extension asked +for; the Firebase CLI grants these for you. + +### Unchanged + +- The indexed collection is still `COLLECTION_NAME` (default `products`), the + input, output and status fields still default to `input`, `embedding` and + `status`, and embeddings are still written as native Firestore vectors. +- Querying by writing a document to `_/index/queries` still works + the same way, with `query`, an optional `limit` and optional `prefilters`, and + the matching document ids written back to the document under `result`. +- `queryCallable` still requires an authenticated caller, still validates its + argument with the same schema, still rejects a `limit` that is not an integer + above zero, and still returns `{ ids: [...] }`. +- `DEFAULT_QUERY_LIMIT` (default 3) and `DISTANCE_MEASURE` (`COSINE`, + `EUCLIDEAN`, `DOT_PRODUCT`, default `COSINE`) behave as before. +- Gemini and Vertex AI embeddings are still `gemini-embedding-001` at 768 + dimensions. +- A custom endpoint still receives `{ batch: [...] }` and must return + `{ embeddings: [[...]] }`, and still requires all three of + `CUSTOM_EMBEDDINGS_ENDPOINT`, `CUSTOM_EMBEDDINGS_BATCH_SIZE` and + `CUSTOM_EMBEDDINGS_DIMENSION`. + ## API surface - **Main entry** (`@firebase/firestore-vector-search`): exports diff --git a/kits/rtdb-limit-child-nodes/CHANGELOG.md b/kits/rtdb-limit-child-nodes/CHANGELOG.md index 66f7e8b76..711eb60d3 100644 --- a/kits/rtdb-limit-child-nodes/CHANGELOG.md +++ b/kits/rtdb-limit-child-nodes/CHANGELOG.md @@ -1 +1 @@ -- Initial release +- Initial release of kit, see README for differences between the legacy extension and this kit diff --git a/kits/rtdb-limit-child-nodes/README.md b/kits/rtdb-limit-child-nodes/README.md index 056a087eb..ce920805e 100644 --- a/kits/rtdb-limit-child-nodes/README.md +++ b/kits/rtdb-limit-child-nodes/README.md @@ -114,6 +114,75 @@ Instance ids must be unique across all kit stanzas in the project, and every instance's function names are namespaced by its `kit--` prefix, so the instances cannot collide. +## Differences from the Limit Child Nodes extension + +This kit is the extension repackaged as an npm package. The trimming logic is +identical: it still watches direct children of one path, counts the parent's +children on every create, and deletes the oldest first until the maximum is met. +Your data is untouched by the move. What changes is the name of one setting, when +bad values are caught, and where the function runs. + +### `NODE_PATH` is now `RTDB_NODE_PATH` + +Node.js reserves `NODE_PATH` for its own module resolution and overwrites it in +the function runtime, so the setting had to be renamed. Copying `NODE_PATH` from +an installed instance's config has no effect: the kit ignores it and falls back +to its default of `messages`, so it watches the wrong path and silently trims +nothing you care about. Rename the key to `RTDB_NODE_PATH` in your `.env`. + +Leading and trailing slashes are now trimmed, so `/rooms/messages/` and +`rooms/messages` are equivalent. + +### `MAX_COUNT` now defaults to 100, and 0 is rejected + +Both settings were required at install; both now have defaults +(`RTDB_NODE_PATH: messages`, `MAX_COUNT: 100`), so an incomplete config deploys +instead of stopping to ask you. `MAX_COUNT` is also a proper integer setting now. +The extension accepted `0`, which meant "delete every child on every write"; the +kit rejects it along with negative and non-integer values. + +### Bad settings surface on the first write, not at install + +The install prompts used to reject a path containing spaces, a non-numeric +`MAX_COUNT` and an invalid database instance id before anything was deployed. +Those checks now run when the function handles its first event, so a bad value +deploys cleanly and then throws on every write to the watched path: + +``` +maxCount must be a positive integer. +``` + +The parent node is not trimmed, and the only sign is the error in your function +logs. + +### `SELECTED_DATABASE_INSTANCE` and the function's region + +`SELECTED_DATABASE_INSTANCE` still defaults to your project's default database, +read from `FIREBASE_CONFIG` rather than injected by the install flow. If your +`FIREBASE_CONFIG` has no `databaseURL`, there is no default and the CLI prompts +for the instance at deploy time. + +The function itself no longer has a location setting. It deploys to your +codebase's default region (`us-central1` unless you have changed it) rather than +the location you picked at install. + +### The trigger is 2nd gen + +`rtdblimit` is a 2nd gen Realtime Database function where the extension was 1st +gen. Its service account needs `roles/eventarc.eventReceiver` and +`roles/run.invoker` on top of `roles/firebasedatabase.admin`; the Firebase CLI +grants these for you. This otherwise only matters if you have alerting keyed to +function generation. + +### Unchanged + +- The trigger fires on creates of direct children of the watched path, and the + parent is trimmed by deleting the oldest children first in a single update. +- Nothing is deleted while the child count is at or below `MAX_COUNT`. +- Errors are caught and logged rather than retried, and the log messages are + the same. +- There are no events to subscribe to; the extension did not publish any either. + ## API surface - **Main entry** (`@firebase/rtdb-limit-child-nodes`): exports `rtdblimit`. The diff --git a/kits/speech-to-text/CHANGELOG.md b/kits/speech-to-text/CHANGELOG.md index 66f7e8b76..711eb60d3 100644 --- a/kits/speech-to-text/CHANGELOG.md +++ b/kits/speech-to-text/CHANGELOG.md @@ -1 +1 @@ -- Initial release +- Initial release of kit, see README for differences between the legacy extension and this kit diff --git a/kits/speech-to-text/README.md b/kits/speech-to-text/README.md index 6a173b634..916ed348e 100644 --- a/kits/speech-to-text/README.md +++ b/kits/speech-to-text/README.md @@ -126,6 +126,103 @@ under the legacy extension id (kept for compatibility with existing consumers): - `firebase.extensions.storage-transcribe-audio.v1.complete` on success - `firebase.extensions.storage-transcribe-audio.v1.fail` on failure +## Differences from the Transcribe Speech to Text extension + +This kit is version 0.1.9 of the extension repackaged as an npm package. The +pipeline is ported closely: the same trigger on finalized objects, the same +ffmpeg transcode to LINEAR16, the same long-running recognition request, the same +per-channel transcript map, the same Firestore progress document and the same two +Eventarc events. Every setting keeps its extension environment variable name and +default, so a `.env` copied from your installed instance needs no value changes. +What changes is where the intermediate audio file is written, how long the +function may run, and what is no longer checked for you. + +### The transcoded copy no longer lands under `tmp/` + +The extension named the transcoded WAV after the local temporary file it had just +written, so with no `OUTPUT_STORAGE_PATH` the copy appeared in your bucket as +`tmp/.wav`, and with `OUTPUT_STORAGE_PATH: transcriptions` as +`transcriptions/tmp/.wav`. The kit names it after the original +object instead: `.wav`, or +`transcriptions/.wav`. + +The transcript itself is written to the same place as before +(`.wav_transcription.txt`, under `OUTPUT_STORAGE_PATH` when set), +so only the intermediate audio moves. If you have lifecycle rules, cleanup jobs +or client code that expect the WAV under a `tmp/` prefix, point them at the new +path. Both files still carry the `isTranscodeOutput` metadata flag that stops the +function from processing its own output. + +### The function may now run for nine minutes + +Recognition is polled to completion inside the function, and the extension ran +with the default 60 second timeout, so long audio failed part way through. The +kit sets `timeoutSeconds: 540`. Memory is unchanged at 1 GiB (`1024MB` in the +extension's terms). + +Temporary files are also deleted after every invocation now. The extension left +the downloaded and transcoded files in `/tmp`, which is shared across warm +invocations of the same instance and counts against the function's memory, so a +busy instance could run itself out of space. + +### Nothing checks your settings at deploy time + +The extension rejected a `LANGUAGE_CODE` that did not look like a BCP-47 code and +a `COLLECTION_PATH` that was not a valid collection path, before it would install. +Neither is checked now. `LANGUAGE_CODE` is still required, so the CLI prompts for +it if it is missing, but any string is accepted and a bad value surfaces as a +Speech-to-Text error per file, with the failure recorded on the Firestore +document and in the `fail` event. + +### The function has no location setting + +`LOCATION` is gone. The function deploys to your codebase's default region +(`us-central1` unless you have changed it) rather than the immutable location you +picked at install. + +### Create the Eventarc channel yourself for events + +Choosing events at install used to create the channel and set both event +variables for you. The kit only reads them: set `EVENTARC_CHANNEL` in your `.env` +to a channel you have created, and the same +`firebase.extensions.storage-transcribe-audio.v1.complete` and `.fail` events are +published. Per-event selection is gone in practice, because the CLI rejects any +`.env` key beginning with `EXT_`, so `EXT_SELECTED_EVENTS` cannot be set and both +event types are published. With `EVENTARC_CHANNEL` unset, nothing is published and +the function is otherwise unaffected. + +### `fail` events for unexpected errors now say what went wrong + +Typed pipeline failures (a zero-stream file, an ffmpeg error, a null +transcription) carry the same payload as before. Unexpected errors did not: the +extension published the caught `Error` directly, and because an `Error`'s +`message` and `stack` are not serialised to JSON, subscribers received +`{"error":{}}`. The kit publishes `{ error: { message, stack } }` instead. + +### The trigger is 2nd gen + +`transcribeAudio` is a 2nd gen Cloud Storage function where the extension was 1st +gen. Its service account needs `roles/eventarc.eventReceiver` and +`roles/run.invoker` on top of `roles/storage.objectAdmin` and +`roles/datastore.user`; the Firebase CLI grants these for you. + +### Unchanged + +- `EXTENSION_BUCKET` still selects the bucket that is both watched and written + to, and still defaults to your project's default bucket. +- `ENABLE_AUTOMATIC_PUNCTUATION` still reads the same `true` / `false` values, and + `MODEL` still defaults to `default`. +- Firestore output is still opt-in: with `COLLECTION_PATH` unset nothing is + written to Firestore, and with it set you still get a document per file created + in `PROCESSING`, moved through `PROCESSING`/`FAILED`, and finished with the + transcription and status. +- Objects with no content type, or a content type that is not `audio/*`, are + still skipped with the same `No content type provided.` and + `Invalid content type.` messages on the Firestore document. +- Multi-channel audio still produces a transcript per channel tag, and a file + with more than one stream still produces a warning rather than a failure. +- There is no backfill for audio already in the bucket, as before. + ## API surface - **Main entry** (`@firebase/speech-to-text`): exports `transcribeAudio`. The diff --git a/kits/storage-resize-images/CHANGELOG.md b/kits/storage-resize-images/CHANGELOG.md index 66f7e8b76..711eb60d3 100644 --- a/kits/storage-resize-images/CHANGELOG.md +++ b/kits/storage-resize-images/CHANGELOG.md @@ -1 +1 @@ -- Initial release +- Initial release of kit, see README for differences between the legacy extension and this kit diff --git a/kits/storage-resize-images/README.md b/kits/storage-resize-images/README.md index 799cc2107..c3898e560 100644 --- a/kits/storage-resize-images/README.md +++ b/kits/storage-resize-images/README.md @@ -133,6 +133,58 @@ When `EVENTARC_CHANNEL` is configured, the function publishes lifecycle events such as `onStart`, `onStartResize`, `onSuccess`, `onError`, and `onCompletion` under `firebase.extensions.storage-resize-images.v1.*`. +## Differences from the Resize Images extension + +This kit is the extension repackaged as an npm package. It is a close port: every +setting keeps its name, type, default and meaning, so an existing `.env` is a +lift-and-shift, and the resizing behaviour, output naming, metadata handling, +download-token regeneration and Eventarc events are unchanged. The differences +below are worth knowing before you deploy. + +### Content filtering runs in the function's region + +When `CONTENT_FILTER_LEVEL` is set (or you supply a `CUSTOM_FILTER_PROMPT`), +the Vertex AI call now uses the region the function is deployed to. The +extension used the region you picked at install time, falling back to +`us-central1`. + +Gemini is not available in every region. If you deploy to a region it does not +serve, filtering fails and the image is treated as a filter error: it is not +resized, and the original is written to your `FAILED_IMAGES_PATH`. Deploy to a +region with Vertex AI support if you use content filtering, or leave +`CONTENT_FILTER_LEVEL` at `OFF`, in which case no Vertex call is made at all. + +### The trigger is 2nd gen + +`generateResizedImage` is a 2nd gen Cloud Storage function, where the extension +was 1st gen. `FUNCTION_MEMORY` still accepts the same values (512 through 8192) +and maps onto the equivalent 2nd gen memory setting. + +The function's service account needs `roles/eventarc.eventReceiver` and +`roles/run.invoker` on top of the roles the extension asked for. The Firebase +CLI grants these for you. + +### Region + +The function deploys to your codebase's default region (`us-central1` unless +you have changed it), rather than a region chosen at install time. See the +content filtering note above, since the two are now linked. + +### Path lists are validated at deploy time + +`INCLUDE_PATH_LIST` and `EXCLUDE_PATH_LIST` must still be comma-separated +absolute paths, but the check now runs when the function loads rather than when +the extension is installed. A malformed value fails the deploy with +`Invalid includePathList: must be a comma-separated list of absolute path +values.` rather than being rejected by an install prompt. + +### No backfill + +There is no function to resize images that already exist in the bucket. The +extension carried the same limitation (its backfill function was disabled), so +this is not a regression, but it is worth stating: only objects uploaded after +you deploy are resized. + ## API surface - **Main entry** (`@firebase/storage-resize-images`): exports