Add one more log level (notice/verbose?) #334
Labels
No labels
bug
duplicate
easy
enhancement
help wanted
invalid
question
rfc
structured-logging
wontfix
No milestone
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
gh-rust-lang/log#334
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
When writing command line programs, I often find myself wanting two distinct "info"-like log levels: one for regular output and one when a
--verboseflag is requested by the user. Then implementing--verboseis simply a matter of changing the log level. I think theDebuglevel is wrong for this purpose because it's oriented towards developers, whereas--verboseis typically intended for troubleshooting by users.There would be two ways to go about it:
Warn > Notice > Info > DebugWarn > Info > Verbose > DebugI realize that similar things have been requested before (#240, #229), but I think my use case is actually a quite helpful one.
If your consideration is to keep the number of built-in log levels low, I actually don't find a distinct
Tracelevel to be useful; seems like debug output to me. But perhaps other people do? What was the justification for that in the first place? :)Hi @intgr 👋
This doesn’t sound unreasonable, however I would still argue against another info-like level for this purpose, because the levels are really arbitrary already any changes aren’t necessary going to result in quality filtering from libraries.
If you’re looking for a way to do nice optional verbose output for your own CLI then you could define whatever semantics you need for the log levels you use (as an example, using something like
env_logger’s filtering to make—verbosemean logging all events raised by your CLI, but only info or higher level events from dependencies).Trace is used often in low-level libraries like hyper and tokio for very verbose logging information that can be useful to diagnose issues.
I think a "Notice" log level and a "Critical" log level would both be very nice to have. I would argue "Critical" in particular is a very necessary addition.
I was recently integrating logging for an application with Google Cloud. They define several more log levels that log does: https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#logseverity
Personally I am thinking that (to an extent) the more levels
logoffers, the better. Libraries that wrap some external logging API can choose how they want to maplog's levels to the API, howeverlogdoesn't currently have enough levels to meaningfully make use of the levels provided in Google's API (or any similar API with more levels thanlogprovides).Google says that "Info" is general runtime info, such as performance stats, while "Notice" would be more significant events, like process startup, shutdown, or config changes. I think this is a meaningful difference that is worth supporting.
In terms of a "Critical" event class, I think this is especially useful (much more than "Notice") for differentiating between "this is an error that by itself isn't a big deal" and "this is an error that by itself is likely impacting my production environment negatively right now". Google offers a few more levels worse than Critical, though I I don't see those as quite as necessary in general (but, again, more levels means logging libraries can have a more fine-grained mapping of levels, and it's not like you are forced to use every log level in your application if you don't want/need them).
I don't think either Notice or especially Critical log levels are that uncommon. I know (for example) Windows supports a Critical log level.
Given this is (AFAIK) the de-facto logging facade for Rust, it is currently disappointing that I can't utilize these additional levels in Google's or anyone else's API.
Adding a log level is a breaking change. The existing levels match slf4j (with the exception of a fatal level for historical reasons).
Breaking changes happen. The de-facto logging crate shouldn't hinder itself by refusing to make breaking changes.
The existing levels clearly do NOT match many other logging APIs. I don't see why a single Java library is considered the authority on logging levels. Why not Google? Or Microsoft? Or Amazon? All three of these entities provide a "Critical" (or "Fatal") log level that is impossible to map from
login a generic and meaningful way. Surely having the ability to better support these platforms is worthwhile.I don't mean to sound hostile (I probably do, sorry), but I don't see either as valid reasons to avoid change.
I also agree that log should have both Notice and Critical. Unix syslog has them and they are very useful. I'm currently writing an app where I'm really feeling the lack of both of these.
Any more movement on this? I could also use some more severity levels.
I think the log crate needs to support the default eight Unix syslog severity levels. See RFC3164 and RFC 5424. Many systems have used these levels as a standard to build upon and/or actually use syslog.
Minimally, please add 'notice'. It's a key level in enterprise applications that's necessary for situations where you want low-noise logs, but need to see certain interesting events. 'Warning' is too strong, meriting attention, whereas 'notice' is something you'd log in case you needed to look back for something that might hint at a problem, but not until you notice something is awry.
For others, here's the overlap between syslog and log:
Having a level between Warn and Info would allow addressing messages
Having a level higher than Error would allow distinguishing messages
In general, I would argue against a proliferation of extra log levels in favor of better filtering on other properties of the log event. Levels are at best a coarse-grained proxy for the likelihood of an event being important to surface/retain. The crate currently offers a reasonable coarse-grained set of levels that cover the majority of cases. Adding more of them doesn't necessarily result in better diagnostics, and having to map them one-to-one to the levels of any other system isn't necessarily a path to a useful set of levels in
log.We've been steadily working towards better structured logging support over the last few years (there is also the
tracingcrate that already supports it) that would better serve users that want to work with some custom log levels, such as an associated syslog level.Sad that there is no custom levels - I'd like to use the:
I wish there as an always level. Sometimes you want to print some stuff like what port and address a web server is running on. I just use warning but it looks kind of weird...
I suppose you could just make a crate and prints with the same format used by the logging system.
I think communicating end-user messages is really orthogonal to
log, which is targeted at collecting diagnostic output. There might be some overlapping technical concerns but the details of end-user rendering I think are different enough that a dedicated domain-specific implementation would be better.I would also love to see a new level added.
External crates especially love to fill up
INFOwith lots of useful messaging but sometimes I find wanting aMSGorNOTICElevel for targeted important messages that don't make sense inERROR/WARN. It is possible to get around this by using crate level log level filters but that's clunky imo, and I often want my scope all encompassing.Custom levels would do some to alleviate this, I haven't looked into it, but would a PR be welcome/well received?
CRITICAL,IMPORTANT,ALERT, andALWAYSall feel like they could be useful.I too would like to see a higher logging level. I've always used
FATALin every language as a hint that this is a potentially irrecoverable error, and in some instances I will set aFATALlog to trigger an automatic panic or shutdown with the log message.I understand this crate also strives to be compatible in
no_stdenvironments, but perhaps an std feature to set dynamic log levels? I've forked the repo to add inFATALand have played with patching my fork in other logging libraries to some success.Any further input I have would be a rehash of the above, but I would like to say that the existing levels truly are fine for serving crude categories of information and simply find a more critical category would be apt. My personal opinion would lean into a fully abstract implementation of all of these to be defined by the consumer (with the existing levels as default), but that would constitute an entirely new paradigm that probably necessitates a separate crate entirely. I may end up exploring this idea using this crate as a base, but the existing crates, like tracing, implement much of what I want even if they're a bit heavy.
As an aside, for the state this crate is in, I agree with @Billy-Sheppard whether
FATALorCRITICALetc is used.Classic log levels can never meet a wide range of needs. For example, Ceph has more granularity, like log level from 1 to 100,they are just number. I don't think as a generical log crate, the classic log levels is enough.
their code looks like this :
dout(20) << func << " waiting for previous aio to complete" << dendl;
dout(30) << "dump:\n";
I would also like to have a
noticeandcriticallevel. I don't think better filtering is a substitute for more log levels. I think following the same spec that unix does seems reasonable to me.To not have a breaking change, it might be a good idea to implement the new/custom log levels behind a non-default feature. This would ensure that existing crates making use of
loghave no breaking changes (for example whenmatching the log level).This way, users can select if they want the existing 5 log levels, or additional. Personally, I think more log levels are useful to distinguish events further.
panic!is not really a replacement forFATAL, as it does not log but instantly stop the application.Apple's swift/log has:
Coming from Swift world, I miss
criticalandnotice.I still fail to understand what is the reasoning behind not adding more levels except it being "breaking change" which I believe would be addressed pretty quickly & easily in the impl. crates.
I understand that some users may find the current set of levels sufficient, but that may not be the case for others, especially in case of mapping levels from other systems as the bijection is often impossible.
I think the best way to figure this out would be a simple poll.
You're vastly underestimating how much effort it takes to update an entire ecosystem to a new version of a crate like log.
Take a look at the downloads of Log v0.3: https://crates.io/crates/log/0.3.9, it still gets 10,000-20,000 downloads a day (in the last 90 days). If you look at it's dependents it still list 16k crates on crates.io: https://crates.io/crates/log/reverse_dependencies.
v0.4 was released 6 years ago.
6 years and it's still being used. And that's not even the main problem. The main problem is (in)compatibility between crates using log. A lot of effort was put into the v0.3 -> v0.4 transition to make it as smoothly a possible, but it's not perfect, same will be true for a possible transition from v0.4 to v0.5 (or v1?).
I don't think we should be making breaking changes lightly, it creates a lot of churn in the ecosystem.
Question is where you draw the line. What levels should be included, what shouldn't and why?
We definitely need a log level between Info and Debug. Right now in a large project, we have a huge trace amount of trace output which is disabled by default. But then even debug is seriously noisy. We have to maintain a large list of targets that need muting for debug to be usable. We are abusing the targets when ideally there would be an extra log level.
Please add notice and critical level. It is very necessary.
The kernel log levels are:
0 (KERN_EMERG) The system is unusable.
1 (KERN_ALERT) Actions that must be taken care of immediately.
2 (KERN_CRIT) Critical conditions.
3 (KERN_ERR) Non-critical error conditions.
4 (KERN_WARNING) Warning conditions that should be taken care of.
5 (KERN_NOTICE) Normal, but significant events.
6 (KERN_INFO) Informational messages that require no action.
7 (KERN_DEBUG) Kernel debugging messages, output by the kernel if the developer enabled debugging at compile time.
Why notice and critical are needed:
🔶 notice
Definition: A level between info and warn.
Use case: Meant for important but non-problematic events. Examples:
System configuration changes.
An admin user logs in.
An event that should be visible to operators but is not a warning or error.
Why info is not enough: info can be noisy—it may include routine operational logs (e.g., thread started, cache warmed up). notice highlights messages that are not errors, but still require attention.
🔴 critical
Definition: A level above error, indicating:
Threats to system stability.
Events requiring immediate action.
Catastrophic failures (e.g., data loss, system corruption).
Why error is not enough: In many applications, error is used liberally (e.g., failed login attempt, network timeout). critical distinguishes events that are truly urgent and severe:
Critical database corruption.
Out-of-memory errors.
Service crashing or halting.
As it's been six years, movement or maintainer comment would be greatly appreciated.
Both @KodrAus and myself made comments on this thread pointing out issues, has there been any progress on fixing these?
What about creating a new crate from a fork of this one? In that crate, add additional levels that:
It would seem to need to be more system environment focused than general purpose as WASI Logging states:
One issue is that we expose
Level's andLevelFilter's repr as usize, and there is no space between each level.It should be possible to leave some space between levels, such as:
Then when we want to add more level like NOTICE/CRITICAL, we won't break ourselves.
Users should not depend on the repr but use them as enum. Ord and Eq still works, but no prev/next.
Refs -
github.com/apache/logging-log4j2@9b9a0d43/log4j-api/src/main/java/org/apache/logging/log4j/spi/StandardLevel.javagithub.com/apache/logging-log4j2@9b9a0d43/log4j-api/src/main/java/org/apache/logging/log4j/Level.javagithub.com/qos-ch/slf4j@183aaa50/slf4j-api/src/main/java/org/slf4j/event/EventConstants.javagithub.com/qos-ch/slf4j@183aaa50/slf4j-api/src/main/java/org/slf4j/event/Level.javaThat actually doesn't matter too much. Because simply adding a variant to
LevelFilteris already a breaking change.The
usizerepresentation should really been an internal detail, not a public one (maybe something to switch for the next breaking change version).I've ended up softening on this issue over the years, and when we do another breaking release (which isn't planned at this stage) I'm on board with adding a level after error for critical/fatal conditions, but not on adding another notice/informational level, which we already have three of.
The need for CRITICAL/FATAL is obvious and won't be a major issue. We're looking forward to it!
However, adding NOTICE is still a major need, and in my opinion, it needs to be addressed...
Currently, trace is treated as syslog debug, debug as syslog info, and info as syslog notice. This leads to significant disruptions in some applications.
I believe that NOTICE should be added.
Maybe we should add the ability to remap these levels to syslog in two modes:
I’m not in favour of adding Notice unless we remodel
Levelentirely to be based on syslog levels, which I’m currently not because it would mean removing a level we already have. The levels in this crate are not a 1-to-1 mapping with syslog, and it’s always possible to rationalise another level between 2 others based on some specific use-cases. So unless we want to anchor levels fully to an existing concept there’s no end to this discussion.i'm in favor of "more than current". i think this crate should make breaking changes extremely rarely. like maybe the next set should be considered the final set forever and always.
what i don't understand however, beyond the stable default set discussion at hand, is, if so many clearly want a different set and it's v hard to converge, clearly different users have different needs. why not make it configurable? perhaps i'm missing something obvious? i do believe there's plenty precedent.
wise lib maintainers will stick with the default. wise app-space users will add whatever levels are convenient.
i see this suggestion was made back in aug 2022 already. is there an obvious reason we can't do this? i believe Rust's third core tenet was "productivity". i appreciate the effort made to find a good default set but agree with some its discussion will likely last forever. but at least for the vast majority of rust users, in app space, it doesn't have to. they could configure their preferred levels. happy to create a new issue for this.
@alextes how do you propose to make it configurable?
i don't think this is the issue to discuss this @Thomasdezeeuw . if you think its within the realm of possibility how about i create a new issue for it?
if you really want to discuss approaches here:
first steps custom level exploration
i don't know this crate well. taking a look for 5m together with AI. two possible approaches:Custom Variant Registry
registry guarded by an RwLock. Add log::register_level(LevelDef) that interns defs and returns a &'static LevelDef, so call-sites can write log!(Level::Custom(MY_VERBOSE), …).
and append registered entries so existing logic keeps its ordering guarantees.
filters.
STATIC_MAX_LEVEL keeps compile-time pruning for the shipped variants, while custom levels are always decided at runtime.
variant and drops the current #[repr(usize)], so this is a “next major” story and needs a migration plan for consumers relying on mem::size_of::() or integer casts.
Metadata Sidecar Level
stores { name: &'static str, ordinal: u16 }. Add MetadataBuilder::custom_level() so backends (or new helper macros) can stamp records with richer level signals while still picking the
nearest built-in Level as the fallback.
macros nothing changes; a new log_custom! (or user-provided macro) can pass a CustomLevel handle together with the fallback Level.
in just keep using record.level() and remain unaware.
new macros or using MetadataBuilder directly. Cons: filtering for the new levels is opt-in and purely runtime; the compile-time max_level_* features cannot prune custom levels, and the
fallback Level has to be chosen carefully so legacy backends don’t mis-route records.
again, v quick reading but seems there's some tough trade-offs around what macros would be available, maybe users with custom levels would have to roll their own or call a non-macro version. another tough trade-off seems to be around performance. what can we prune at compile time. i feel one would have to dive deeper to call those. like maybe there is a real perf hit, maybe not, for log probably any hit is unacceptable, but then would you still let app users set a mode where they take the hit, or suggest to create another logging crate for this entirely to wrap
log. idk. tough trade-offs. would have to explore and discuss together.Your suggesting as a solution for this issue so I think place is fine to discuss it. If you want to create a new issue that's also fine.
I'm not going to read an AI answer though, I have no interest in that. Please consider other people time and don't waste it. If you have a though through proposal that's great, but letting AI generate something isn't that.
We shouldn't blindly look only at syslog. But it certainly has additional levels that we don't have: CRITICAL (something worse than ERROR) and NOTICE (something between WARNING and INFO).
And in some applications, these additional levels are very necessary. I believe they should be added as soon as possible. They can always be flattened if the log destination doesn't support our granularity.
The point of using exactly syslog's levels would be to sidestep any discussion on where to draw the line on adding more levels to
log.The issue with an extensible system is that it would make any level not directly mapped by
logsecond-class. If we were to explore it, I think something like this might be the best:So you can set a custom level and get it, but can always reduce it down to a regular
Level. That also avoids us needing to figure out how to order custom levels with predefined ones.If a breaking change will be introduced, couldn't it use numerical levels and add utility to clamp values to predefined levels?
For example, standard levels are modeled after syslog with 10 steps in between. 0=Critical, 10=Error, 20=Warning, etc.
Or it can use 100 steps if the repr can be larger than u8.
Numerical levels are pretty common and it allows customization.
It might even be possible to make it ABI compatible with the current version. Just reserve the already used numbers.
I was thinking the exact same thing. Using steps of 10 should be enough. E.g:
That's similar to what Go's
log/slogpackage does.One problem I've had in the past with numerical log levels is that two systems can disagree what level a numerical value actually is, e.g. what does
30mean? It makes string options a lot safer as no (sane) system would parse"critical"as"info"if it didn't have acriticallog level (like log currently). However, if we go with (public) numerical values we lose the option to convert it to string in all cases.I would also not want to introduce numerical levels. Log levels are already z-indexes, but that’s largely offset by their semantics and coarse granularity. I don’t want to open the door to discussions like should this log be a 102 or a 103. This also leaves the problem of components needing to agree on what numbers any levels between levels, such as Notice, mean if they want to interpret them.
Having said that, if we were to introduce numerical log levels though, I think we could adopt the same scheme as OpenTelemetry: https://opentelemetry.io/docs/specs/otel/logs/data-model/#field-severitynumber
That leaves just a few values per level for intermediate severities, and is already compatible with our current set of levels (plus a critical/fatal level).
Ref -
I'm recently reading Go's
log/slogpackage and would consider a similar design a possible way to go:Other logics may be similar to
log/slogThe overall logic should be able to be adapted into our current
Leveldesign. I'll try to implementlogforth's Level in this way to demonstrate the idea.UPDATE: However, I noticed that opentelemetry-rust defines the severity as an enum:
github.com/open-telemetry/opentelemetry-rust@c54d2b84d0/opentelemetry/src/logs/record.rs (L141-L230)(Out of the thread, I borrowed a new
LevelFilterdesign from spdlog that may also be helpful, but it should be a different thread if anyone is interested in it.)IMHO, log levels to use and their semantics should be specific to applications, and crates like
logshould provide a sufficient set of levels so that applications can choose the ones they want to use.In case it is helpful, perhaps one could take a look at the "glog" API, e.g. as implemented by https://abseil.io/docs/cpp/guides/logging or https://github.com/golang/glog.
There are named log levels (FATAL, ERROR, WARNING, INFO, DEBUG), but also "verbose" logs which use an integer to denote verbosity. To avoid misunderstandings about what verbose log levels mean, verbose logging can be turned off or on per module, as well as for everything. (I also think that named log levels are given negative log level values, while verbose logs ended up being positive, but that's a detail.)
I personally found this approach rather useful over the years: have a few commonly-agreed upon logging levels, and then a large sea of "verbose" logs with arbitrary level of detail. This avoids needing to litigate the log levels that something should be logged at. It also avoids needing to agree on the "correct" meaning of log levels across modules or programs. And in the end allows the user to provide debug information with varying levels of granularity, when needed.
Fatal/Critical log level is a must here.
Coming from .NET ecosystem, i don't get how that could be an oversight for such sort of essential log level.
Regarding automation of business processes fatal/critical log level usually means some unrecoverable error that requires immediate SRE/System Administrator intervention (event itself might be extremely expensive for business).
Currently it looks like the basic logging facade is designed not accurately enough (i would say even carelessly).
Just for reference how it looks in .NET ecosystem.
LogLevel from MS Logging Facade (please note the
Criticallog level):Serilog's LogLevel enum definition (.NET's most popular 3rd-party logging provider, please notice the
Fatalvariant, it is aligned one-to-one withCriticalfrom logging facade):So, i am voting for fatal variant.
Looking at previous comments the majority of different stacks have well aligned logging levels in their ecosystem, except of Rust.