InfraStudio Agents

InfraStudio Agents

Dogfooding: a billing-webhook finding to a merged PR

A real change to InfraStudio Agents, with repository evidence, an observed timeline, successful checks and the review's remaining concerns.

Last updated

The finding: a late event could restore an old plan

InfraStudio Agents uses its own agents on its own product. This case follows one concrete result: PR #155, opened by InfraStudio Agents for finding F12 from issue #90. It addressed a billing-webhook problem recorded as BRK-6. The PR, its review and its checks live in our own repository, which is not public. The screenshot and narrative below summarize those records, with the exact timings taken from repository metadata.

Stripe subscription events were applied in arrival order. A delayed active subscription update arriving after a cancellation could give the workspace its paid plan back. An older plan update could similarly overwrite a newer plan. Event deduplication alone cannot prevent that: two different legitimate events can arrive in an order that does not represent the latest state.

The finding described a correctness problem in the control plane, not an analytics conversion opportunity. We use it to demonstrate the finding-to-PR part of the workflow, rather than claiming that it proves every source integration or every agent succeeds. No customer information or customer result is part of this case.

GitHub shows the InfraStudio Agents bot’s billing-webhook pull request merged by a person, with four checks and six changed files.
A cropped GitHub screenshot of PR #155, captured 8 November 2026. Select the image to view it at full size. It shows the bot author and human merge; exact UTC timings below come from repository metadata. Account navigation and unrelated page content are outside the crop.

The approved plan and implementation

The PR records the approved plan for F12 and identifies the resulting implementation commit, 3b7a3852ba3cec7e9f8b6d6efddab785ffca24df. The chosen approach read the subscription back from Stripe for non-delete subscription events, instead of trusting an older event body. A timestamp guard was an alternative, but would have required stored state and a database migration outside this change's scope.

The patch added a helper for fetching the live subscription and changed the webhook handler to use that result for creation and update events. Cancellation events continued to apply their own body, so a cancellation did not require another API request. An unknown subscription returned a recorded acknowledgment without applying a plan; other Stripe API failures followed the hook's retry behavior.

The PR describes six changed files, with 56 added and nine deleted lines. Unit tests and adversarial webhook checks were adjusted to exercise late updates and unknown subscriptions. The existing strict expected-failure marker for BRK-6 was removed when the relevant attack passed. That is a useful test of a specific regression, not a proof of every possible concurrent delivery order.

The observed timeline on 5 November 2026

  • 20:42:05 UTC: PR #155 opened.
  • 20:42:12 UTC: the four recorded CI checks began.
  • 20:46:01 UTC: the last of those checks completed successfully.
  • 20:50:24 UTC: the automated review was submitted with no blocking issues and several non-blocking observations.
  • 20:57:05 UTC: a person merged the pull request.

The interval from opening to merging was 15 minutes. That is observed open-to-merge time, not total engineering time, time saved or an end-to-end agent runtime. It excludes the earlier discovery, issue discussion, plan preparation and any work before the PR opened. The timestamps do not establish how much attention a human spent reviewing the change.

The recorded check durations were test: 1 minute 52 seconds, breaker: 3 minutes 49 seconds, runner-image: 2 minutes 22 seconds and terraform: 17 seconds. These checks ran in parallel, so adding their durations would misrepresent elapsed CI time. They are observations from this PR, not a runtime promise for another repository or change.

The review did more than issue a verdict

The automated review on commit 3b7a385 found no blocking issues, but recorded meaningful caveats. It pointed out that a subscription lookup returning 404 could be acknowledged without a diagnostic log. It also asked for a test of the subscription-event API failure path, so a later refactor would not accidentally record a failed fetch as handled and prevent a retry.

The review also called out a known concurrency window: an update could fetch the current state just before cancellation, then commit just after the cancellation event applied. That limitation was already described in the PR. Closing it fully required a broader state-ordering change. This case does not claim that PR #155 eliminated every webhook race, and it does not present later work or an unmerged fix as part of this result.

There were smaller observations about consistency, documentation wrapping and shared test state as well. A no-blockers verdict means the reviewer judged the scoped change acceptable with those caveats. It does not mean no risks existed. The human merge decision remained responsible for weighing the scope, the evidence and the follow-up work.

What we measured, and what we did not

Token usage and token cost are unavailable for this case. We do not estimate them from the model name, elapsed time or patch size. The platform supports recording usage and computing an estimate with operator-configured provider rates, but that capability is not a substitute for an actual measurement attached to this historical run. There is no verified provider bill here to quote.

We also have no controlled comparison against doing the same work manually, so this case makes no return-on-investment or productivity percentage claim. A merged PR is an observable engineering artifact. The successful checks support the assertions they exercised; they do not establish that production was free from all billing defects afterward, or that a business metric improved because of the change.

How to evaluate the same loop in your project

Start with a small, well-evidenced finding and an explicit boundary for implementation. Review the proposed plan, keep protected changes outside the agent patch, and read the diff together with the automated review. Inspect check results on the exact commit you intend to merge. Keep caveats visible and create separately scoped follow-up work when they deserve it. That makes the result reviewable even when some measurements are missing.

Read analytics to pull request for the broader signal-to-code workflow, the PR reviewer for an actual inline comment, the CI fixer for an example of a limited diagnosis, and error triage for the Sentry investigation process. The homepage, pricing and security overview explain how to begin an evaluation with your own repository and AI key.

Evaluate the workflow on your repository

Choose your model, inspect the evidence, and decide what ships.

Get started View pricing