The last post in this series ended with an unflattering number. After the Change Feed pushes moved work from the Container App to the Function app, the Container App dropped to about a cent a day, and the Function app went up to €7.57 for the month. Of that, €5.95 was Log Analytics data ingestion. The code I had added cost a few cents a day to run. What cost money was the telemetry around it.
This post is about finding out what that telemetry was, and about the three tries it took to stop it.
What I assumed, and what the data said
I assumed it was my own log lines. A function that fires every 5 minutes, one every 15, an hourly one and a Change Feed trigger on every write: that has to produce a lot of lines saying nothing is wrong.
The workspace has a Usage table that says what was actually ingested, per table. For September 25 to October 25 (2,377 MB):
| Table | Ingested | Share |
|---|---|---|
| AppMetrics | 1,717 MB | 72% |
| AppTraces (the log lines) | 306 MB | 13% |
| AppPerformanceCounters | 300 MB | 13% |
| AppRequests, AppExceptions, AppDependencies | 54 MB | 2% |
My log lines were 13%. Nearly three quarters was metrics, and I had never configured a metric. The daily volume followed the functions I added: about 35 MB a day in late September, 57 in early October, 104 after October 16 and 129 from October 20 to 24. At about €2.50 per GB, that is the €5.95.
The query:
Usage
| where TimeGenerated >= datetime(2026-09-25) and IsBillable == true
| summarize MB = round(sum(Quantity), 1) by Day = bin(TimeGenerated, 1d), DataType
| order by Day asc, MB desc
Try 1: drop the metrics in my own code
The names in AppMetrics were runtime and web server meters: dotnet.gc.*, dotnet.jit.*, kestrel.*, http.server.*, process.*, and azure.functions.health_check.reports. My Program.cs sets up OpenTelemetry for the worker, so I added a view that drops every metric instrument before export, switched off the exporter's performance counters and standard metrics, and tested in a scratch project that the drop view really exported nothing. Merged, deployed.
Nothing changed. AppMetrics kept coming at 6,000 to 9,800 rows an hour, the same as before.
What I should have looked at first
A Functions app is two processes: the host (the Azure Functions runtime) and the worker (my code). Every metric row carries the SDK version of whoever sent it. Querying by name and SDK version:
AppMetrics
| where TimeGenerated > ago(14h)
| summarize Rows = count() by Name, SDKVersion
| order by Rows desc
Every row said azurefunctions:4.1054.0.0_dotnet10.0.13-servicing.26:otel1.15.3. That is the host, with its own OpenTelemetry (1.15.3), not my worker, which uses 1.18. The rate per metric name before and after my deploy was the same. My drop view was correct and could never have mattered, because it lived in the other process. The column that said so had been there all along.
Try 2: the standard environment variable
Microsoft's documentation says the Functions host supports the standard OpenTelemetry environment variables. The standard "no metrics exporter" setting is OTEL_METRICS_EXPORTER=none, so I set it as an app setting. About 15 to 20 minutes later, the metrics were still arriving at about 500 rows per 5 minutes. As far as I could measure, it does nothing for the host's metrics. I kept it in the infrastructure file, as a harmless extra, but I'm not claiming it does anything.
Try 3: leave the host's OpenTelemetry mode
My host.json had "telemetryMode": "OpenTelemetry". That is what puts the host in OpenTelemetry mode, and the host then exports its own runtime metrics regardless of what the worker does. I removed the line, so the host falls back to the classic Application Insights integration, which doesn't send those meters. Merged and deployed on October 26 at 12:21 UTC.
The rows went from about 500 per 5 minutes to about 40, from about 13:25 UTC, once the old instance had gone.
| Day | Total | AppMetrics | AppTraces | Perf counters |
|---|---|---|---|---|
| Oct 23 to 25 | about 130 MB | 90 to 93 | 17 to 19 | 15.7 |
| Oct 26 (changed mid-day) | about 98 MB | 63.5 | 20.1 | 13 |
| Oct 27 | 48.9 MB | 10.9 | 20.9 | 16.5 |
| Oct 28 | 47.9 MB | 10.8 | 20.0 | 16.4 |
Two full days at about 48 MB, against about 130 before: a reduction of about two thirds. In money, at about €2.50 per GB, the Function app's workspace goes from about €0.32 a day to about €0.12 (my estimate from the megabytes, not the Cost analysis figure, which lags). What remains is my own log lines (42%), the performance counters the classic integration sends (34%) and a little metric volume (23%). I could trim those too, but it is worth about €0.07 a day, and I left it.
The quick fix I didn't take: a daily cap
There was a much faster way to cut the bill. A Log Analytics workspace has a daily cap, and the minimum you can set is 0.023 GB, about 23 MB a day. That limits ingestion to at most about €0.06 a day, which is less than the €0.12 I ended up at. I'd have been done in two minutes, without three tries.
I didn't take it as the fix, for these reasons:
- A cap cuts the day off, it doesn't choose what to keep. Before the fix the workspace received about 130 MB a day, so a 23 MB cap would be reached after roughly four hours. Everything after that is dropped: the traces, the requests and any exception from the moderation function. Even at 48 MB a day it would be reached around the middle of the day.
- What gets kept is mostly the noise. Three quarters of the early data was runtime metrics nobody reads. The cap would have filled up on those and thrown away the logs I do use, for the rest of every day.
- It hides the failure when it matters. A broken function after the cap is hit leaves no trace. I'd be paying less to be blind for most of the day.
- The source keeps sending. The host still builds and exports every metric and the cap only throws the data away at the workspace. Fixing the source also saves the sending, and it left the numbers in the tables meaningful.
A cap is a good safety net and a bad fix. It stops the worst day, but you should treat hitting it as an alarm, not as the plan. I did lower the cap on the workspaces, to 0.5 GB a day, well above the normal 48 MB, so it only triggers when something is really wrong.
What it cost me
Leaving OpenTelemetry mode has side effects:
- Dependency telemetry stopped. I can no longer see which Cosmos or HTTP call inside a function was slow or failed. It was about 1% of the volume, and the traces still carry errors from the code itself. I accepted that.
- Less correlation between host and worker. The documentation says OpenTelemetry mode in both gives better correlation. For functions this small I haven't missed it yet.
- Performance counters moved to the classic integration, which sends them slightly more often than before.
I checked that the Function itself still works: I posted a test comment, the moderation function scored it (the LLM score showed up on the comment), I approved it, and the comments push put it on the page from KV about 90 seconds later.
The real lesson: I found this by accident
I found this while writing the previous post in this series, the before/after comparison of the Change Feed pushes. The Function app was supposed to cost €0.92 a month and the cost export said €7.57. To explain that number in the article I queried the workspace's Usage table, and only then saw where the money went. I was looking for material for a post, not monitoring anything. Without that, the Function app would have kept costing about €13 a month for a blog that nothing depends on, and I would have noticed when my Azure credit ran out.
What I'd do differently:
- Look at log intake every week, and especially right after deploying anything new. The
Usagequery above takes a minute, and the per-day, per-table view is the one that shows a change in shape. - Look at who sent the data before fixing anything. The
SDKVersioncolumn would have saved me my first try. - Test the mechanism, then check the result in the data. My drop view worked in a scratch project and did nothing in production, because the scratch project tested my code and the production data came from somewhere else.
- Treat telemetry defaults as a cost decision. They are tuned for visibility, not for a small bill.
- Put a daily cap on the workspace as a safety net, not as the fix. A cap won't tell you something changed, and it drops the rest of the day's logs once it is hit. Set it well above the normal volume, so that hitting it means something is wrong.
Where it stands
- The Function app's log ingestion is down from about 130 MB to about 48 MB a day (two full days measured, Oct 27 and 28), and the weekly check is now on my list.
- The next full-month comparison, once the Function side has had a month at the new rate, goes at the bottom of this series.