Short answer

Fireworks AI has disclosed two intrusions. In early October an attacker read environment variables, including stored secrets, from evaluation jobs in a limited number of accounts. On October 24 an attacker reached development machines and exposed account and billing information and, for a limited set of accounts, secrets, datasets and incomplete model weights. Fireworks says customers who received no notice need no incident-specific action. If your app calls its API, find every place the key lives, move it out of the iOS app and behind your own proxy, rotate your keys and the secrets you stored in Fireworks, check usage for LLMjacking, and set a spend alert.

Fireworks AI, the inference platform behind a lot of open-model features in apps and coding tools, has now disclosed two separate intrusions in about three weeks. Its first bulletin, published October 9, said an attacker used internal Fireworks credentials to read environment variables from evaluation jobs in a limited number of customer accounts. The update dated November 9 goes further: account and billing information was exposed, and for a limited set of accounts the attacker also reached secrets, datasets and incomplete model weights.

If your app or your backend sends requests to api.fireworks.ai, this post is the checklist I would run today. It covers what Fireworks has said so far, how to tell whether you are affected, how to rotate keys without breaking your app, what to look for in your usage, and why an AI key should never sit inside an iOS app bundle in the first place. Everything here comes from Fireworks' own bulletins and documentation, linked in the Sources section at the end.

What Fireworks has said so far

Fireworks' two bulletins tell one story in two stages.

Stage one came in early October. On October 6 at 9am Pacific, Fireworks identified unauthorized use of internal credentials and started containment. Customer notifications began on October 7. On October 9 Fireworks rotated employee API keys, told the customers it had identified, and published a public bulletin that night. On October 10 it revoked more user API keys and rotated service-account tokens after a credential review. That bulletin says the attacker reached environment variables used in evaluation jobs, including API keys and tokens customers had stored as secrets. At that point Fireworks said it had found no evidence of access to customer models, datasets, training data, prompts or outputs, and that it does not store prompts or outputs by default.

Stage two came in late October. The November 9 update says a third-party cloud provider that hosts Fireworks' R&D machines had its own security incident on October 23. On October 24 an unauthorized party used that to reach Fireworks development machines and ran a workload that exposed more customer data. Fireworks says it caught this the same day and has not detected further successful access to the customer platform since October 25.

The update lists three kinds of data involved:

  • Resource metadata: resource identifiers, display names, descriptions and creation times.
  • Billing information: addresses, invoice and payment history, account balance and credit usage. Fireworks says it does not store payment card numbers, so those were not affected.
  • Model weights and datasets: for certain customers, a subset of incomplete model weights and datasets.

Fireworks says Inference, the Training SDK and Nexus were not affected according to findings so far, and notes that they are zero data retention by default. It has turned off self-serve dataset uploads and downloads for Batch Inference and dataset-based training while the investigation continues. It has brought in Palo Alto Networks Unit 42, notified law enforcement, and links the activity to a financially motivated group that sells stolen LLM access to illicit routing gateways, which the industry calls LLMjacking.

Forbes reported on November 9 that two customers it spoke to had moved much of their business away, and that Fireworks counts about 10,000 enterprise customers. That part is reporting, not something Fireworks' bulletin states, so treat it as context.

The timeline in one table

The two bulletins cover a month of events. Laid out by date, with everything taken from Fireworks' own account, they look like this.

Date (2026)What Fireworks says happened
October 6Unauthorized use of internal credentials identified at 9am Pacific. Containment starts.
October 7Customer notifications begin.
October 9Employee API keys rotated, identified customers told, public bulletin published that night.
October 10More user API keys revoked and service-account tokens rotated after a credential review.
October 23A third-party cloud provider that hosts Fireworks' R&D machines has its own security incident.
October 24An unauthorized party reaches Fireworks development machines and runs a workload that exposes more customer data. Fireworks says it caught this the same day.
October 25The date after which Fireworks says it has detected no further successful access to the customer platform.
November 9Update published. It adds billing information, resource metadata and, for certain customers, datasets and incomplete model weights.

Two dates matter for your own checks. A key that existed before October 10 and was never rotated has lived through both incidents. And any usage review in Step 6 should start before October 6, so you have a normal week to compare against.

Step 1: Work out whether you are affected

The first bulletin is direct about this. If you did not get a notice from Fireworks, it has not identified your account as affected and no incident-specific action is required. If you did get one, the notice lists the secret names involved, and Fireworks says it can send secret names and credential fingerprints over a secure channel.

So start with your inbox and the inboxes of everyone on the account. Search for mail from Fireworks since October 7, and check the address on your billing profile too, since that is often a finance alias nobody reads.

Even without a notice, I would still do steps 2, 5 and 6 below. They take an hour, they cost nothing, and an investigation that is still marked "Active monitoring" can widen. The second bulletin already added data categories the first one did not mention.

Step 2: Find every place your Fireworks key lives

You cannot rotate a key well until you know everywhere it is used. A single key can end up in a backend's environment, a CI secret, a local .env file, an .xcconfig added for testing, and sometimes the shipped app itself.

This script lists every match in a repo's working tree, every commit in history that added or removed a match, and every file inside a built .app bundle that contains the endpoint or the variable name. It was tested against a sample repo and a fake app bundle, and it uses only grep, git, find and strings, which ship with macOS and Xcode's command line tools.

#!/usr/bin/env bash
# fw-key-scan.sh: list every place a Fireworks key or endpoint shows up
# Usage: ./fw-key-scan.sh /path/to/repo [/path/to/MyApp.app]
set -uo pipefail
REPO="${1:-.}"
APP="${2:-}"
PATTERN='api\.fireworks\.ai|FIREWORKS_API_KEY|fireworks'

echo "== Working tree: $REPO"
grep -RInEi --exclude-dir=.git --exclude-dir=DerivedData \
  --exclude-dir=Pods --exclude-dir=node_modules --exclude-dir=build \
  "$PATTERN" "$REPO" || echo "   no matches"

if git -C "$REPO" rev-parse --git-dir >/dev/null 2>&1; then
  echo "== Git history (commits that added or removed a match)"
  git -C "$REPO" log --all --oneline -G "$PATTERN" -i || true
fi

if [ -n "$APP" ] && [ -d "$APP" ]; then
  echo "== Built app bundle: $APP"
  find "$APP" -type f -print0 | while IFS= read -r -d '' f; do
    if strings -a "$f" 2>/dev/null | grep -qEi "$PATTERN"; then
      echo "   match in: ${f#"$APP"/}"
    fi
  done
fi

Point the second argument at the .app inside an archive (in Xcode's Organizer, right-click the archive, choose Show in Finder, then Show Package Contents and go into Products/Applications). A match there means the endpoint or key name was compiled into what you ship. The endpoint alone is harmless. A key string next to it is not.

Write down each location in a short table: where the key is, which environment it serves, and who can change it. You will use that table in step 4.

Step 3: If the key is inside your iOS app, move it out first

AI-built apps often call a provider straight from the phone, and the Fireworks incident is a good reason to fix that. A key placed in Swift code, an Info.plist, an .xcconfig or a bundled JSON file ends up in the app on every user's phone. Anyone who downloads the app can pull the bundle apart and read strings out of it. That is exactly the kind of credential LLMjacking groups resell, and it means rotating the key forces an app update and leaves everyone on the old build broken. Claude Code's "Don't Ask Again" Can Save Your Passwords covers another way a key ends up in a repo.

The fix is a small server that holds the key and forwards only the requests you allow. Here is a minimal Cloudflare Worker that does that. Its request handling was tested locally against a mocked upstream: it rejects unauthenticated calls, ignores whatever model the app asks for, and caps max_tokens.

// Cloudflare Worker: the app calls this, only this holds the Fireworks key
export default {
  async fetch(request, env) {
    if (request.method !== "POST") {
      return new Response("Method not allowed", { status: 405 });
    }
    const url = new URL(request.url);
    if (url.pathname !== "/v1/chat") {
      return new Response("Not found", { status: 404 });
    }

    // Replace this with a real check: your own signed session token,
    // or an App Attest assertion verified on the server.
    const auth = request.headers.get("Authorization") || "";
    if (!(await isValidAppSession(auth, env))) {
      return new Response("Unauthorized", { status: 401 });
    }

    let body;
    try {
      body = await request.json();
    } catch {
      return new Response("Bad JSON", { status: 400 });
    }
    if (!Array.isArray(body.messages) || body.messages.length > 40) {
      return new Response("Bad messages", { status: 400 });
    }

    const upstream = await fetch(
      "https://api.fireworks.ai/inference/v1/chat/completions",
      {
        method: "POST",
        headers: {
          "Content-Type": "application/json",
          "Authorization": `Bearer ${env.FIREWORKS_API_KEY}`,
        },
        body: JSON.stringify({
          model: env.FIREWORKS_MODEL,           // the app never picks the model
          messages: body.messages,
          max_tokens: Math.min(Number(body.max_tokens) || 512, 1024),
        }),
      }
    );

    return new Response(upstream.body, {
      status: upstream.status,
      headers: {
        "Content-Type":
          upstream.headers.get("Content-Type") || "application/json",
      },
    });
  },
};

async function isValidAppSession(authHeader, env) {
  // Placeholder so the sample runs. Do not ship this as is.
  return authHeader === `Bearer ${env.DEV_ONLY_TOKEN}`;
}

Store the key with wrangler secret put FIREWORKS_API_KEY so it never sits in your repo, and set FIREWORKS_MODEL to the full model path you use, in the accounts/fireworks/models/... form shown in Fireworks' quickstart. The endpoint and the Authorization: Bearer header come straight from that quickstart.

Two parts of this sample matter more than the rest:

  1. The placeholder check has to go. A shared token baked into the app is only a little better than the key itself. Use your own user sessions if your app has accounts, and look at Apple's App Attest (in the DeviceCheck framework) if it does not. Apple describes App Attest as a way to make sure requests your server receives come from legitimate instances of your app.
  2. The server picks the model and caps the output. If someone does get past your check, they can only run the one model you chose, at a bounded size, which keeps a stolen session from turning into an open-ended bill.

Ship the app version that talks to your proxy before you rotate, if you can. Then the rotation in step 4 only touches your server.

The app side of the proxy

The Worker in Step 3 is half of the change. The other half is the code in your app that used to call Fireworks directly. It gets simpler, because the app no longer knows the provider, the model or the key. It sends messages to your endpoint with the user's session token and maps the status codes to a few states the UI can show.

import Foundation

struct ChatMessage: Codable {
    let role: String
    let content: String
}

enum AIError: Error {
    case signedOut   // 401: your session check said no
    case paused      // 503: you switched the feature off on the server
    case busy        // 429 or another 5xx: keep the user's text and offer a retry
    case badReply
}

struct AIClient {
    let endpoint: URL
    let sessionToken: () async throws -> String   // your own login or App Attest flow

    func reply(to messages: [ChatMessage]) async throws -> String {
        var request = URLRequest(url: endpoint)
        request.httpMethod = "POST"
        request.timeoutInterval = 30
        request.setValue("application/json", forHTTPHeaderField: "Content-Type")
        request.setValue("Bearer \(try await sessionToken())", forHTTPHeaderField: "Authorization")
        request.httpBody = try JSONEncoder().encode(Body(messages: messages, max_tokens: 512))

        let (data, response) = try await URLSession.shared.data(for: request)
        guard let http = response as? HTTPURLResponse else { throw AIError.badReply }
        switch http.statusCode {
        case 200: break
        case 401: throw AIError.signedOut
        case 503: throw AIError.paused
        case 429, 500...599: throw AIError.busy
        default: throw AIError.badReply
        }
        let decoded = try JSONDecoder().decode(Completion.self, from: data)
        guard let text = decoded.choices.first?.message.content else { throw AIError.badReply }
        return text
    }

    private struct Body: Encodable {
        let messages: [ChatMessage]
        let max_tokens: Int
    }

    private struct Completion: Decodable {
        struct Choice: Decodable { let message: ChatMessage }
        let choices: [Choice]
    }
}

This was compiled with Swift 6.2.3 on macOS and run against a local mock server that answered 200, 401, 503 and 429 in turn. It returned the reply text for the first and threw signedOut, paused and busy for the others, and a 200 with an unexpected body surfaced as a decoding error. It was not run inside an iOS app or against the Worker itself, so wire it to your real session token and test it on a device.

Three choices in it are worth keeping when you adapt it. There is no model name and no provider URL in the app, so changing either is a server change. The request has a timeout, because an AI call that hangs for a minute feels like a frozen screen. And 401 is handled separately from the other failures. After you rotate keys, a 401 from your own proxy means the user's session is the problem and never the provider key, which makes support tickets much easier to read.

Give the AI feature an off switch

A proxy also lets you turn the feature off without shipping an app update. That is useful during a week like this one: while you rotate keys, while a provider has paused part of its service, or on the day your usage check shows traffic you cannot explain. Add a setting on the server and return a clear status when it is off:

    // Add this at the top of fetch(), before the auth check.
    if (env.AI_ENABLED === "false") {
      return new Response(JSON.stringify({ error: "ai_paused" }), {
        status: 503,
        headers: { "Content-Type": "application/json", "Retry-After": "3600" },
      });
    }

Those lines were run under Node 22 with AI_ENABLED set to "false", "true" and unset. They returned 503 with the JSON body in the first case and passed the request through in the other two. In the app, treat 503 as "the feature is paused": hide the button or show a short message, and keep whatever the user typed. Set the variable with wrangler secret put AI_ENABLED or as a plain environment variable in your Worker's settings, and try the switch once before the day you need it.

Step 4: Rotate keys without breaking your app

Fireworks' advice to affected customers is to create new keys, update the applications that used the old ones, and expect anything still holding a revoked key to fail authentication until it is updated. The order below keeps downtime close to zero. All commands are from Fireworks' firectl reference.

List what exists. An admin can see keys for every user on the account:

firectl api-key list --all-users

Look for keys you do not recognize. Fireworks asks customers to report unfamiliar names or IDs, because the attacker created keys of their own during the first incident.

Give each system its own identity. Fireworks' docs recommend service accounts for backends and CI instead of personal keys, so automated calls show up separately in audit logs. Create one per system with the narrowest role that works. For a server that only runs inference, that is inference-user, which can view resources and run inference but cannot create or change anything.

firectl user create --user-id "ios-app-proxy" --service-account --role=inference-user
firectl api-key create --service-account "ios-app-proxy" --key-name "proxy-2026-11" --expire-time "2027-05-10"

Keys expire one year after creation by default and cannot be set longer than 365 days, so an explicit, shorter expiry is a reminder to rotate on a schedule rather than after the next breach.

Deploy the new key, confirm traffic, then delete the old one. Update the secret in your proxy, CI and any other location from your step 2 table. Make one real request through each path. Only then delete the old key:

firectl api-key delete <old-key-id>

If you keep personal keys for local development, rotate those too, and check that nobody's laptop still has an old key in ~/.fireworks/auth.ini, which is where firectl reads its account ID by default.

Step 5: Rotate the secrets you stored inside Fireworks

The first incident was about exactly this kind of secret. If your evaluation jobs needed a GitHub token, a database password or another provider's API key, you may have stored it as a Fireworks secret. List them:

firectl secret list

Fireworks' bulletin makes a point that is easy to misread: removing a stored secret does not invalidate the credential behind it. Deleting the secret from Fireworks stops Fireworks from holding a copy, and the token stays valid wherever it was issued. For each secret:

  1. Rotate it at the provider that issued it (GitHub, your database host, the other AI provider).
  2. Update your applications to use the new value.
  3. Update or delete the Fireworks secret.
  4. Review the access logs on that provider's side for anything done with the old credential. Fireworks recommends rotating even if you find nothing.

For iOS teams, the secrets worth thinking about here are App Store Connect API keys and signing-related tokens if your evaluation pipeline ever touched release tooling. Those are high-value. If one was in a Fireworks secret, revoke it in App Store Connect and issue a new one. Backdoored Claude Code Extension in Cursor lists the iOS keys to rotate after a compromise.

Step 6: Look for LLMjacking in your usage and resources

LLMjacking means someone else runs inference on your account and you get the bill. The signals are a spend jump, models you never call, and resources you did not create. Three checks cover most of it.

Daily usage by model. Pull the last two months grouped by model and look for any model name you do not use, or a day that stands out:

firectl billing get-usage --start-time 2026-09-01 --end-time 2026-11-11 \
  --usage-type serverless --group-by model_name --timezone Asia/Jakarta

Run it again with --usage-type dedicated-deployment if you have deployments. Set --timezone to your own zone so the daily buckets line up with when your users are awake.

Users and resources. Fireworks asks customers to check for unrecognized API keys, users, evaluators, training jobs and datasets. Start with accounts, including service accounts:

firectl user list
firectl user list --filter 'service_account=true'

Remove anyone who has left the team, and any service account nobody can explain.

Audit logs, if you are on Enterprise. Fireworks' audit logs record reads, writes and deletes on storage. Search for create events and for activity from key IDs you did not issue:

firectl audit-logs list --start 2026-10-01 --filter 'message:"Create"'
firectl audit-logs list --start 2026-10-01 --no-paginate -o json > fw-audit.json

The CLI works in whole UTC days. The REST endpoint at /v1/accounts/{account}/auditLogs accepts ISO 8601 times if you need tighter windows, and Fireworks' docs warn that an empty page with a nextPageToken means the scan is still running, so keep paging until the token is gone.

Step 7: Put a ceiling on future damage

Rotation deals with this incident, and a spend alert limits the cost of the next one. Fireworks lets you set monthly spend thresholds that notify you when crossed:

firectl billing notification-settings update --monthly-spend-usd-thresholds=200,500

Set the first threshold a little above a normal month, so the alert arrives while there is still time to act.

Beyond Fireworks, the same pattern holds for every AI provider your app uses:

  • One key per system, so you can revoke one without touching the rest.
  • Keys only on servers, behind a proxy that fixes the model and caps tokens.
  • Short expiry dates and a calendar reminder to rotate.
  • Spend alerts on every provider account, sent to a person who reads them.
  • Secrets for third-party services stored with the least access possible, and never assumed safe because a vendor holds them.

Step 8: If your datasets held user data

The November 9 update says that, for certain customers, a subset of datasets was accessed. If you uploaded datasets built from your users' content (chat logs used for fine-tuning, support tickets, uploaded documents), and your notice says your datasets were involved, this becomes a privacy question as well as a security one.

This is not legal advice, and the rules depend on where you and your users are. The practical developer steps are these: find out exactly which datasets your notice lists, what fields they contained, and whether they could identify a person. Then read your own privacy policy and the privacy answers you gave in App Store Connect and check they still describe what you do. Take the facts to whoever handles legal questions for your company before deciding on any notice to users. Fireworks says it is notifying impacted customers directly with recommended response actions, so that notice is the starting document.

Going forward, strip or pseudonymize personal data before it leaves your servers for any training or evaluation provider.

Step 9: Check what the paused uploads break

While the investigation runs, Fireworks has disabled self-serve dataset uploads and downloads for Batch Inference and dataset-based training. If you have a nightly batch job or a fine-tuning pipeline that uploads data, it will fail. Check your scheduled jobs now rather than finding out from an alert at 3am. The Training SDK remains available, according to the update, so that may be a path for training work that cannot wait. For batch inference, the short-term choice is to run requests through the regular inference API with your own queue, or to pause the job until Fireworks restores uploads.

A one-hour checklist

  1. Search every inbox on the account for a Fireworks incident notice since October 7.
  2. Run the scan script on each repo and on your latest archived .app.
  3. If the key is in the app, ship a build that calls your own proxy instead.
  4. Create service accounts with inference-user and new keys with an expiry date.
  5. Deploy new keys everywhere, test, then delete the old ones.
  6. List Fireworks secrets and rotate each one at its issuer.
  7. Check daily usage by model, users, service accounts and (on Enterprise) audit logs.
  8. Set monthly spend alert thresholds.
  9. If your datasets held user data and your notice mentions them, collect the facts for your legal contact.
  10. Check scheduled jobs that depend on dataset uploads.

FAQ

I did not get a notice. Do I need to do anything?
Fireworks says no incident-specific action is required if you were not notified. Rotating keys, checking usage and setting spend alerts are still cheap habits, and the investigation is still marked as active monitoring.

Were my prompts and outputs exposed?
Fireworks' bulletins say Inference was not affected according to its findings, that Inference and Nexus are zero data retention by default, and that it does not store prompts or outputs by default. The findings are described as findings to date.

Were payment cards taken?
Fireworks says it does not store payment card numbers and they were not affected. Billing addresses, invoices, payment history, balances and credit usage were in the exposed data.

Is deleting the secret in Fireworks enough?
No. Fireworks states that removing a stored secret does not invalidate the credential. Rotate it with the provider that issued it.

Should I leave Fireworks?
That depends on your risk tolerance, your contracts and how Fireworks handles the rest of the investigation. Whatever you decide, the steps above apply to any provider you move to.

Who do I contact at Fireworks?
The bulletins say to reply to your incident notification email or write to security-notice@fireworks.ai (the November update) or security@fireworks.ai (the October bulletin).

If you build iOS apps with Claude Code and want it to treat keys, signing and personal data carefully, I put together Claude Code + Xcode: The Production iOS Workflow Kit. It is a set of Claude Code templates for SwiftUI and UIKit projects: 6 commands, 10 agents and 8 rules (one of them for privacy and security), with risk classification and human approval gates around auth, payments, personal data and signing, plus a runnable Xcode sample. It guides the tool; it does not guarantee security, so you still review every plan and diff.

Sources