Skip to content

Reduce default ec2net syslog noise; add optional debug logging#140

Closed
mjnowen wants to merge 1 commit intoamazonlinux:mainfrom
mjnowen:patch-2
Closed

Reduce default ec2net syslog noise; add optional debug logging#140
mjnowen wants to merge 1 commit intoamazonlinux:mainfrom
mjnowen:patch-2

Conversation

@mjnowen
Copy link
Contributor

@mjnowen mjnowen commented Mar 17, 2026

Summary

Reduce default logging from amazon-ec2-net-utils so normal operation produces fewer ec2net lines in syslog, while keeping important info/error messages and making full debug output available when needed.

Problem

On systems using the package (e.g. Ubuntu with systemd-networkd), ec2net logs many lines on every refresh: IMDS token and metadata queries, config file reuse, and "No addresses found" for the primary interface. That happens on every timer run and can dominate syslog, making it harder to spot real issues.

Changes

  1. Optional debug logging

    • debug() in lib/lib.sh only calls log debug "$@" when EC2_NET_UTILS_DEBUG is set (non-empty).
    • Default: debug messages are not logged.
  2. Documentation and default at top of lib.sh

    • Short comment explains that debug is controlled by EC2_NET_UTILS_DEBUG.
    • Optional commented line # EC2_NET_UTILS_DEBUG=1 for temporary local debugging.
    • : "${EC2_NET_UTILS_DEBUG:=}" so the variable is defaulted when unset; if systemd (or the script) sets it, that value is kept.
  3. "No addresses found" reclassified

    • In create_rules(), the message is changed from info() to debug() and uses ${iface} instead of ${ether}.
    • This case is normal for the primary interface or when there are no extra addresses; treating it as debug avoids routine noise while keeping it available when debugging.

Behaviour

  • Default: Only info (e.g. "Starting configuration refresh", "Reloaded networkd", lock retries) and error are logged.
  • With debug: Set EC2_NET_UTILS_DEBUG=1 (or any non-empty value) via systemd (e.g. Environment=EC2_NET_UTILS_DEBUG=1 in a unit override) or by uncommenting the line in lib.sh for local runs. All current debug messages (IMDS, config reuse, "No addresses found", etc.) are then logged.

Testing

  • Default: run refresh without setting EC2_NET_UTILS_DEBUG; syslog should show only info/error from ec2net.

Fixed:

2026-03-17T16:02:55.577210+00:00 ip-172-31-20-194 systemd[1]: Starting refresh-policy-routes@ens5.service - Refresh policy routes for ens5...
2026-03-17T16:02:55.583538+00:00 ip-172-31-20-194 ec2net[3791]: Starting configuration refresh for ens5
2026-03-17T16:02:55.630911+00:00 ip-172-31-20-194 systemd[1]: refresh-policy-routes@ens5.service: Deactivated successfully.
2026-03-17T16:02:55.631035+00:00 ip-172-31-20-194 systemd[1]: Finished refresh-policy-routes@ens5.service - Refresh policy routes for ens5.
  • Debug: set EC2_NET_UTILS_DEBUG=1 for the service or in the script and run refresh; previous verbose ec2net lines should appear again.

With EC2_NET_UTILS_DEBUG=1 enabled, the syslog now looks like this:

2026-03-17T16:07:27.732222+00:00 ip-172-31-20-194 systemd[1]: Starting refresh-policy-routes@ens5.service - Refresh policy routes for ens5...
2026-03-17T16:07:27.738558+00:00 ip-172-31-20-194 ec2net[3933]: Starting configuration refresh for ens5
2026-03-17T16:07:27.748598+00:00 ip-172-31-20-194 ec2net[3933]: Got IMDSv2 token for interface ens5 from http://169.254.169.254/latest via ens5
2026-03-17T16:07:27.750518+00:00 ip-172-31-20-194 ec2net[3933]: [get_meta] Querying IMDS for mac
2026-03-17T16:07:27.758128+00:00 ip-172-31-20-194 ec2net[3933]: [get_meta] Querying IMDS for mac
2026-03-17T16:07:27.766952+00:00 ip-172-31-20-194 ec2net[3933]: Using existing cfgfile /run/systemd/network/70-ens5.network
2026-03-17T16:07:27.768617+00:00 ip-172-31-20-194 ec2net[3933]: [get_meta] Querying IMDS for mac
2026-03-17T16:07:27.775048+00:00 ip-172-31-20-194 ec2net[3933]: [get_meta] Querying IMDS for mac
2026-03-17T16:07:27.783159+00:00 ip-172-31-20-194 ec2net[3933]: [get_meta] Querying IMDS for network/interfaces/macs/0a:82:bb:80:2a:b9/local-ipv4s
2026-03-17T16:07:27.792906+00:00 ip-172-31-20-194 systemd[1]: refresh-policy-routes@ens5.service: Deactivated successfully.
2026-03-17T16:07:27.793063+00:00 ip-172-31-20-194 systemd[1]: Finished refresh-policy-routes@ens5.service - Refresh policy routes for ens5.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Reduces repetitive IMDS/config messages in syslog while keeping operational info and errors. Debug can be enabled via systemd Environment= or in-script.

- Gate debug() on EC2_NET_UTILS_DEBUG so debug messages are off by default

- Document EC2_NET_UTILS_DEBUG at top of lib.sh (env or uncomment for local debug)

- Reclassify "No addresses found" from info to debug (routine on primary/refresh)

- Keep info/error unchanged; enable full debug with EC2_NET_UTILS_DEBUG=1
@joeysk2012
Copy link
Contributor

Feel like some of the debug logs like Got IMDSv2 token for interface ens5 via ens5 are useful to us. Can you make that an info ?

@mjnowen
Copy link
Contributor Author

mjnowen commented Mar 20, 2026

Feel like some of the debug logs like Got IMDSv2 token for interface ens5 via ens5 are useful to us. Can you make that an info ?

Hmmm, this is exactly the kind of log messages we are trying to not silence, just quieten via the debug flag. Nothing stopping a tester or developer from simply enabling the debug flag permanently in their environment if a debug message is useful for their specific context. So, I acknowledge what your suggesting, but I want to pushback on this and ask for evidence beyond a "feel like some". Use-cases for why you need this log message displayed all the time for all customers? (I'm not saying no, I'm just asking for data to backup this suggestion).

@joeysk2012
Copy link
Contributor

Seems like we should be using built in syslog functionality rather than an enivornment variable. These logs are not needed 99% of the time but It does helps us track down race conditions live and catches them when they happen.

@mjnowen mjnowen closed this Mar 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants