Privacy Policy

This Privacy Policy explains how Benchwright LLC (“Benchwright,” “we,” “us,” or “our”) collects, uses, shares, and protects information about you when you use benchwright.ai and our related services (the “Service”). By using the Service, you agree to the practices described here.

1. Who we are

Benchwright LLC is a Delaware limited liability company that operates an autonomous benchmarking platform for evaluating large language models and AI systems. Our registered address is:

Benchwright LLC
2810 N Church St, PMB 507205
Wilmington, Delaware 19802-4447
United States

For privacy questions, write to legal@benchwright.ai.

2. Information we collect

2.1 Information you give us

• Account information. Email address, password (hashed), and, if you sign in with Google or GitHub, the profile fields those providers return (name, avatar URL, provider user ID).
• Profile details. Display name and avatar URL, if you set them.
• Billing information. Payment is processed by Stripe. We do not store your card number, CVV, or full bank details on our servers. We store a Stripe customer identifier, the amount and timestamp of each top-up, and your balance, spending cap, and auto-refill preferences.
• User-supplied credentials. If you give us API keys or secrets for third-party services (for example, LLM provider keys for models you want benchmarked), we hold them so we can inject them only as you instruct. Secret values are kept in a separate, access-restricted table that our website and API front ends have no read access to; only the run pipeline can retrieve a value, and only in order to inject it into a run you asked for. Access is further limited per account by database row-level security, the values travel over TLS, and the underlying storage is encrypted at rest by our infrastructure provider. We do not add a further layer of application-level encryption on top of those controls. You can delete a stored credential at any time.
• Benchmark inputs. The benchmark descriptions, datasets, task configurations, and system prompts you submit to run against a model.
• Operator messages. If you use the operator (an agent that runs your eval workflows on your behalf), the messages you exchange with it — including any sent over WhatsApp when you use Benchwright’s messaging number — and the instructions you give it.

2.2 Information we generate when you use the Service

• Run traces & events. For every benchmark run we store a trace record (status, model, costs, token counts, duration, error messages, halt flags) and an append-only stream of events (phase transitions, sandbox lifecycle, cost accrual).
• Model inputs & outputs. The prompts, completions, and intermediate tool calls produced while the pipeline runs against the model you are benchmarking. These are stored as part of the run record so you can review, re-run, and audit results.
• Outbound network logs. For runs that use our sandbox network policy, we record which hosts were reached or blocked, without recording request or response payloads.
• Billing ledger. A history of credits, debits, top-ups, and refunds on your account.

2.3 Information collected automatically

• Log and diagnostic data. IP address, user-agent, request paths, timestamps, and error details, kept in short-term server and edge logs to secure the Service and debug issues.
• Cookies & similar technologies. We use a small number of first-party cookies to keep you signed in (bw_jwt), to count visitors ourselves (bw_vid), and to remember your light or dark theme choice (bw_theme). We do not run third-party analytics or advertising trackers, and we do not share traffic data with any vendor. See our Cookie Notice for details.
• Page views. We count traffic ourselves, recording the page visited, the referring site if your browser sent one, and basic browser information. When you are signed in, this is associated with your account. It is kept for 90 days and never shared with an analytics vendor.

3. How we use information

We use the information we collect to:

• Provide, maintain, and improve the Service, including running your benchmark pipelines, operating the operator on your behalf when you use it, billing you for usage, and surfacing results on your dashboard;
• Authenticate you and keep your account secure;
• Respond to your support requests and communicate service-related notices;
• Detect, investigate, and prevent fraud, abuse, security incidents, and violations of our Acceptable Use Policy;
• Debug and improve reliability (for example, diagnosing a failed run from its event stream);
• Comply with legal obligations, enforce our Terms of Service, and protect the rights, property, and safety of Benchwright, our users, and others.

4. AI training & customer content

We do not use your private account data, benchmark inputs, run traces, prompts, model outputs, or stored credentials to train any model. The only content we may use for model training is content you explicitly publish to a public surface of the Service (for example, a benchmark you choose to publish to a public registry) and content that is already publicly available on the internet.

Third-party model providers you select (for example, OpenAI, Anthropic, Google) have their own training and data-retention policies that apply to the prompts and completions we relay to them on your behalf. You should review those policies directly; where a provider offers a “no training” API mode, that election is between you and the provider.

5. How we share information

We share personal information only in these cases:

• With subprocessors who run parts of our infrastructure (hosting, database, payments, sandbox execution, model inference, email, and operator messaging). See the Subprocessor list.
• With model providers you choose. When you run a benchmark that calls an LLM provider, we send the prompts and any credentials you configured to that provider so the request can be fulfilled.
• With law enforcement or other parties when we believe disclosure is required by law, legal process, or is necessary to protect the rights, property, or safety of Benchwright, our users, or others.
• In a business transfer. If Benchwright is involved in a merger, acquisition, financing, reorganization, bankruptcy, or sale of assets, personal information may be transferred as part of that transaction; we will notify you and, where applicable, give you an opportunity to object.
• With your consent for any other purpose disclosed at the time we collect the information.

We do not sell personal information, and we do not share personal information for cross-context behavioral advertising.

6. Subprocessors

The current list of subprocessors, what data they process, and where they operate is maintained on our Subprocessors page. We review subprocessors before onboarding them and require appropriate contractual and security commitments.

7. Data retention

We keep personal information for as long as we need it for the purposes described in this policy. Typical retention periods:

We may keep information longer if required by law or to resolve disputes, enforce agreements, or protect against fraud or abuse.

8. Security

We use industry-standard measures to protect personal information, including TLS in transit, encryption at rest of the underlying storage by our infrastructure providers, per-account row-level security in the database, keeping user-supplied secret values in a separate table that our website and API front ends cannot read, role-based access control, audit logging, and least-privilege service credentials. We do not apply an additional layer of application-level encryption to stored secret values. No system is perfectly secure; if you believe your account has been compromised, contact us immediately at legal@benchwright.ai.

9. Your rights and choices

Depending on where you live, you may have some or all of the following rights:

• Access. Request a copy of the personal information we hold about you.
• Correction. Ask us to correct inaccurate or incomplete information.
• Deletion. Ask us to delete your account and associated personal information, subject to retention we are required or permitted to keep (for example, billing records).
• Portability. Receive the personal information you have provided in a machine-readable format.
• Restriction / objection. Ask us to pause or limit certain uses of your information.
• Withdraw consent. Where we rely on consent, you can withdraw it at any time.

To exercise any of these rights, email legal@benchwright.ai from the email address on your account, or contact us from an account control where we can verify your identity. We will respond within the timeframes required by applicable law.

10. EU / UK users (GDPR)

If you are in the European Economic Area, the United Kingdom, or Switzerland, Benchwright acts as the “controller” of personal information about you that we collect on our own behalf, and as a “processor” for content you provide when you run benchmarks.

Lawful bases. We process personal information on one or more of these bases: performance of a contract with you (running the Service), our legitimate interests (security, fraud prevention, improving the Service in ways that do not override your rights), compliance with legal obligations, and your consent where required.

You have the right to lodge a complaint with your local supervisory authority. You may also contact us at legal@benchwright.ai to request a data processing addendum.

We do not currently target the Service at residents of the EU/UK but we serve users who sign up from those regions. Where we transfer EU/UK data outside those regions, we rely on the European Commission’s Standard Contractual Clauses or equivalent UK mechanisms.

11. California users (CCPA/CPRA)

If you are a California resident, you have the following rights in addition to those above:

• Right to know the categories and specific pieces of personal information we have collected about you, the sources, the business or commercial purpose, and the categories of third parties we share it with.
• Right to delete personal information, subject to legal exceptions.
• Right to correct inaccurate personal information.
• Right to opt out of the sale or sharing of personal information. We do not sell or share personal information as those terms are defined under California law.
• Right to limit use of sensitive personal information. We only use sensitive personal information (such as account login credentials and stored API secrets) to perform the Service you requested and for security and fraud prevention, which is a permitted use under California law without a separate opt-out.
• Right to non-discrimination for exercising any of these rights.

You may exercise these rights by emailing legal@benchwright.ai. You may designate an authorized agent to act on your behalf; we will ask for written authorization and verify your identity.

12. International data transfers

Benchwright is based in the United States. Personal information we collect may be stored and processed in the United States and in any other country where we or our subprocessors operate. Where required by law, we use appropriate safeguards — such as the EU Standard Contractual Clauses — to transfer data across borders.

13. Children

The Service is not directed to, and we do not knowingly collect personal information from, anyone under 18. If you believe a minor has provided personal information to us, contact legal@benchwright.ai and we will delete it.

14. Changes to this policy

We may update this Privacy Policy from time to time. If we make material changes we will notify you by email or by posting a prominent notice in the Service before the changes take effect. The “Last updated” date at the top of this page will always reflect the current version.

15. Contact

Questions, requests, or complaints about this Privacy Policy or our data practices:

Benchwright LLC
Attn: Privacy
2810 N Church St, PMB 507205
Wilmington, Delaware 19802-4447
United States
legal@benchwright.ai