The title is a bit clickbaity, but ignore that and it's a useful tip you might not know. The author doesn't actually demand you stop using tail -f if you like it better.
I use tail -f and when I need to search, I use iTerm's search (Cmd+F) which highlights matches and even supports regex. I also have an unlimited scroll buffer so I can just scroll up or down to get the context of a particular match.
That only works if you only care about search results since you started tailing. If you want to search for what happened before you started tailing, you need to stop tailing and use less/grep/etc.
This only works if what you are searching for has scrolled past since you started following. If you want to see other occurrences in the file you're out of luck. For example, let's say you are watching to see which error message is generated when something goes wrong, then you want to see when else that error message has been thrown.
multitail is great. I really cannot use command line utils that don't support colors/highlighting anymore. htop instead of top, etc. It just gives me much better visibility and readability. I'm surprised that so many non-color utils are still being used. It just feels so 1994 to me to stare at a white on black display. I guess there's nothing more slow moving and conservative than shell interfaces, thus articles like these that shame us into using different tools.
Personally, I'd love to see some hot young talent just do a 100% redo of the standard gnu utils from an interface perspective. Just go crazy with new interface and display ideas, novel presentation modes, novel navigation, etc while still maintaining backwards compatibility. I could see a big disruption here.
I use htop and turn the colours off... thing I like about htop is I can use the mouse. It's so F-key-based, which, especially on a laptop, means you have to take your eyes off the screen to find the keys.
Of course, I'm not opposed to colours, I'm opposed to colours that look tacky / l33t hax0r, which would be most terminal stuff. I like 𝐛𝐨𝐥𝐝 better than trying to find colours I won't hate.
Switching between many programs and systems, I have a hard time keeping track of what colors mean what. Especially when people start customizing their colors. There are enough tools to show you what a file is, I don't need directories printed in red and files in blue and symlinks in green. ls -l is good enough and works everywhere.
I wrote a Python script (called synesthesia)[1] that'll colorize input based on regex matches, and any matched text will be the same color for the same content.
The use case that drove its development was needing to keep track of UUIDs across multiple logs - and grep --color will colorize its matches, but not differentiate between ones that have different content. With this, I could watch both logs as data was passed from one to the other and keep track of e.g. the orange one.
I also thought it would be nice to be able to use patterns from logstash's grok, so I wrote grokpat[2] to find patterns for me. A lot of grok's patterns use atomic groups, which aren't available in Python 2, so I wrote redi[3] to convert them from grok's syntax to Python compatible syntax.
This _almost_ touches on the useful part of this:
If you search in less and then put it in follow mode, it continues to highlight the search terms. This is very useful for trapping an exception or webcall in the wild.
The downside:
less buffers and tail -f prints directly. On a slow printing log this can cause events to show up slowly, and can cause performance issues on a fast moving log.
If you're piping through a script for processing, tail -f is still the best bet. If you need multiple files, multitail is probably better.
less +F hits a quick+easy operational niche and is available almost everywhere (whereas multitail is not).
But how do you combine the two on the command line, so less is tailing the file AND highlighting a pattern when launched?
I typically read logfiles with less +F, but it would be nice to create an alias for patterns in specific types of logs, so I don't have to remember them or rely on command history.
From creating vi (nvi, on NetBSD) macros, I'm used to thinking in terms of competely replicating the keystrokes that one would do interactively... so I tried it both ways (with and without ^M). The way I published is the one that works. Why the ^M ? Because if you're working interactively the search-pattern isn't submitted until you press Enter. nvi(1) (what NetBSD (and Free and OpenBSD) uses) will accept "-c" "command" arguments which are similar to the less(1) "+"... so, you can:
$ vi -c 123 ./myfile
and start editing "./myfile" at line 123. Nice for edit/compile/edit/compile dance that might happen if you're developing software. Play with that (and try your imagination with other ideas).
The other useful thing is filtering - on sufficiently modern versions of less [1], you can use & to filter the lines, in the same way you use / to search them - only lines matching the filter are shown, and the display continues to update. Use &! for a negative filter. This has replaced grep | tail for me, with the advantage that it's non-destructive, so you can undo the filter, reapply a different one, etc.
[1] Everything i've used on Linux in the last few years, not the vanilla one on the Mac, but the one in Homebrew
I was going to mention multitail as well, which is a fantastic utility. That and rsstail, both by Folkert Van Heusden, are quite slick. The mark it leaves (a red line across the screen) is even more visible than those given by less.
I've commented a bit more on them (in the context of other RSS tools) here:
Though not quite applicable to logfiles, I've written a script to tail my Newsbeuter feed URLs with Multitail support, for a console / terminal-based RSS feed.
Even when you don't want less +F, a good alternative to tail -f is tail -F. It survives log rotation, and the file being temporarily inaccessible:
--retry
keep trying to open a file even if it is inaccessible when tail
starts or if it becomes inaccessible later; useful when follow-
ing by name, i.e., with --follow=name
-F same as --follow=name --retry
Also, if you are using a proper ssh client, opening a second ssh connection to manage the reacting to the log files is habit for me at this point. The only time I'm using tail -F is after a new configuration deployment. Otherwise, I'm looking at archival data in ELK.
Yes. I use it when I need to resume a SSH session. Given proper configuration, switching SSH tabs/connections is as easy as switching browser tabs...there is no real need for me to use screen for such a purpose.
Do not stop when end-of-file is reached, but rather to wait for
additional data to be appended to the input. If the file is re-
placed (i.e., the inode number changes), tail will reopen the
file and continue. If the file is truncated, tail will reset its
position to the beginning. This makes tail more useful for
watching log files that may get rotated. The -f option is ig-
nored if the standard input is a pipe, but not if it is a FIFO.
So, no -F or --retry but a different default behavior
If you want to be pedantic, no, it's not a bug but in the vast majority of the software world people use the term “bug tracker” to refer to a system which also tracks new work which isn't strictly a defect.
And, since we're being pedantic, when I said “vast majority of the software world”, that includes the OpenBSD project:
“Sending in bug reports
If possible, use the sendbug(1) command to get the bug into our tracking system. Sendbug requires that your system can properly send Internet email. If you cannot use sendbug on a functional OpenBSD machine, please send your bug report to bugs@openbsd.org.
Perhaps what you are sending in is a feature request, not necessarily a bug. New features are accepted, especially with code that implements your suggested new feature.”
The idea is that if everyone else is converging on "tail -F" supporting that as an alias means that people's habits & shell scripts just work without modification. That seems worth adding a single line to a switch statement to me.
Why would they? It's not needed - '-F', '--retry' and the like are redundant. IMVHO, OpenBSD's '-f' behaviour should have been the default elsewhere, too. How many times in the past have you deliberately followed the file descriptor (coreutils' default '-f' behaviour) instead of its name? Every single time I used it in the past on Linux, this is what I thought:
"For f***** sake! I should have used '-F'! Arghhh...!"
I'm not saying that the OpenBSD behaviour is wrong but that compatibility is a worthwhile goal. Adding a simple -F alias for their existing behaviour would allow people's habits & shell scripts to just work without changes.
I was introduced to less +F a while ago and it is quite nice, but there is one simple "feature" of tail -f that I miss quite a lot: being able 'mark' the log with gaps by hitting enter a few times. This is especially handy when you have to first load a page or warm up the app before performing the operation you're interested in watching, letting you separate the earlier output from the lines generated by what you're testing.
With less, you have to keep track of the current position by timestamp or other unique message, and it's easy to lose your place when the output starts streaming in. With tail, you only need a moment to mark your spot and then it's visually distinct even as more message come in.
GP isn't meaning it like that, they mean being able to add some spaces so that the next log entries stand out for quick visual identification as you're making changes and reloading the application.
Sure it does. If your log lines are distinct (e.g. they have timestamps or unique IDs) then you can use less's search highlighting to provide a visual marker for a specific line, similar to what you can do by manually inserting a bunch of blank lines on the console.
This trick doesn't work if your log file has a bunch of identical lines and you want to keep an eye on their rate, though.
I won't argue that for this specific use case, tail isn't friendlier than less.
But the original poster posted a useful tip, and is now getting aggressive downvotes and comments like "This has nothing to do with what is being discussed in this subthread." I think that's unwarranted.
Having to remember & type a timestamp has much more mental overhead (planning & memory) than "scan/scroll back to last block of vertical whitespace".
That's why suggestions of either named-marks or back-searches aren't considered equally-attractive alternatives to marking the scrollback with a batch of <return>s.
An example of some of the vi commands that less(1) has adopted. You can also:
* page forward and backward, including using numeric prefixes to indicate a lineno or indicate "repeat 'n' times".
* launch vi if less(1) is working on a file (versus (eg) stdout of some process)
* search fwd/backward
* start examining a new file w/o leaving less(1)
* ...
If you wanted to grab 10 lines from /var/log/messages, starting at the first line that has "pjungwlr" in it, you could:
$ less /var/log/msgs
/pjungwlr^M
^G (note line informational line numbers)
v (launch vi)
[double check your line #s, etc]
:d1,. (delete from 1st line, to current line)
10j (go down 10 lines)
:d,$ (delete to EOF)
:w my_newfile
:q
As a junior programmer I got chewed out by my IT department because I was examining a (production) log file in vi. The sysadmin told me I should use less (actually more---this was on Solaris in ~2001). To this day I only read log files with less, but I've never figured out his objection. Negatives to using vi I can imagine are:
- I might write to the log file. That seems like a real worry, although I could also say `vi -R` to prevent it.
- Starving the production system of memory. I'm pretty sure he expressed this concern. Any insight into whether it is legit? Is less actually any better?
Obviously you really should have a log shipping & aggregation service so you can read logs offline, etc, but not every project is large enough for that, nor every org organized enough. So for the sake of argument my premise is, "Assuming you want to read a log on production . . ."
On large files less fires up immediately and vim is quite slow. I think that vim is loading an entire file in the memory while less just loads visible chunk. May be other vi implementations are more efficient with large files.
Besides — vim really shouldn't be used on log files. It's editor, not viewer.
> Besides — vim really shouldn't be used on log files. It's editor, not viewer.
I think the two concerns I stated are probably legit, but this one is boring to me. Vim is nicer than less for reading files. I have more navigation commands. I can yank part of the file and save it somewhere else. I can pipe a range of lines through awk/etc. I can switch between files. I can split my screen. Etc. Some of these are probably available in less too (more than more(1) supported in 2001), but I doubt all, and I already know the commands in vim. I'm interested in not clobbering my logs and not crashing the server, but if you tell me vim is an editor not a viewer, I'll ask Why?
Btw re-reading my words I don't mean to sound combative. But the point of my original question was to understand. I've been cargo-culting "use less for logs" for 14 years already.
You can save a part of the file, pipe a range of lines, or switch between files in less, too, and the commands are mostly the same as in vi. It'd sure be nice to be able to split the screen, though!
Frequently, frequently I am in the position that I need to manipulate an overly-verbose log file to condense out the information I want. I could spend a half hour concocting wizard-like shell invocations, OR I could do it interactively in five minutes with vi...
If I'm logging into a server later to resolve an issue, one thing I will probably check is the command history. If you open a file with "vi", how do I know that you only looked at it, and did not make any changes to it? When I see "less" in the command history, I know for certain that it was a read-only operation.
I don't know his exact reasoning, but the biggest thing I can think of you already touched upon: there's more of a danger that you can write to a file accidentally with vi vs with less.
Beyond that, I would probably correct a junior employee as well. Even if there's nothing wrong with it, it's not the right tool for the job. When I first started I got 'yelled' at for checking to see if a machine was on the network using tracert instead of ping. It works, but it's not the right tool for the job.
Yes, but cpu is normally the problem. vim is very inefficient with long lines, combine that with all of vim's features (syntax highlighting, etc.) and you have the potential to make a problem worse.
I would never "yell" at anybody for using vi vs less, but I might curiously ask their reasoning behind it. I'm generally for less bikeshedding and cargo cultism when possible. Whatever works to get the job done.
I think your idea of `vi -R` or `vim -R` is a good idea. `less` uses less memory and may load faster but looking at logs for things like valgrind vim will give you syntax highlighting that I am not sure you can get with `less`.
I have used `view` but it didn't give me syntax highlighting. On my system (archlinux) view is actually a symlink to ex (which must change its behavior based on its name?).
I think Edix's answer is the most pragmatic.
Also, someone at work actually crashed a service by opening the log files in vim.
I think the problem was that the memory and processor were already getting stomped on (thus the need to look at the logs) and vim tried to do a lot of fancy stuffs to get more info on the file as a whole.
1) for user: less chance (pun intended) to actually change the file when all you wanted was to read it
2) for sysadmin: if sysadmin sees "less somefile.log" in bash history, he knows the user just read the log. If he sees "vi somefile.log" then he doesn't know if the user has also changed the log file (maybe not even knowing it).
The assumption is that you deal with non-malicious users who just make mistakes (which is often the case).
Thanks for your reply! Since I do a lot of devops I've tried over the years to formulate a mental model of how a sysadmin thinks and what they care about. I'd never have thought about this auditability concern, so it's something to add to my list! :-)
for sysadmin: if sysadmin sees "less somefile.log" in bash history, he knows the user just read the log. If he sees "vi somefile.log" then he doesn't know if the user has also changed the log file (maybe not even knowing it).
In case you didn't know, you can invoke an editor from within less by pressing 'v'. And that wouldn't get registered in the shell history ;)
It was there previously? Sorry I missed it. In any case, I wouldn't call the action I mentioned malicious. There is no hack involved, no improvisation either. Simply using a built-in feature of `less`.
Display only lines which match the pattern; lines which do not match the pattern are not displayed. If pattern is empty (if you type & immediately followed by ENTER), any filtering is turned off, and all lines are displayed. While filtering is in effect, an ampersand is displayed at the beginning of the prompt, as a reminder that some lines in the file may be hidden.
If you're using screen/tmux as myself, tail -f remains a good option, as you can enter gaps or search, or do 1000 more things as compared to plain terminal session
But really, it's 2015. Are you really using a terminal emulator without a buffer search? Are you not using screen or tmux, both of which can do these things natively, with no external tools? Why? And what is wrong with backgrounding the tail command, running your grep, and then foregrounding the tail command?
Personally? The reason I'm not using screen is that there's no way to atomically [attach, create new window] in screen and I value my gnome-terminal tabs.
If tmux can do that, then I'd consider it reason to switch.
Do you mean creating a new window every time you attach to an existing session? Or do you mean that each of those actions ("attach" and "create new window") should be atomic?
Check out 'byobu'. Stupid name, but it's built on screen and can use the screen commands. Byobu has F2 for new terminal tab, F3/4 to shuffle between them.
Depending on what you mean by "atomic" (do you really mean atomic, or do you just not want to have to type anything after you've attached to create the new window?), you can do this with screen's -X option. If that's what you want to do I can try to dig up the way I used to do this.
While you may not be able to do transforms on the output via `sed` or `awk`, you can get a grep-like filtering by using the `&` character with `less` open.
I use tail -f with tmux usually. Will less +F save the entire output into memory? In other words, if I accidentally leave it open in a tmux shell and the log grows to 300mb of output, will the server start swapping?
I usually run `less` niced and with a ulimit of 30 megabytes, but `less` also has an option to configure how much buffer space to use. As far as I can tell, the default is "all of your memory", which is a suboptimal default in my book.
One advantage that tail -f has is that you can insert newlines or other characters into the output to help you visually split log entries. Log files often look the same so being able to visually split up the log is very helpful IMO.
There was a time when tail didn't know inotify. It was so, so much better when it learned that trick.
There was also a time when there was a utility called inotail, bridging that gap until tail proper was improved. I very much preferred it over regular tail.
I always thought it was such a shame that you couldn't use plain poll() for this. poll()-ing on a read fd at EOF for a regular file should work like poll()-ing a network socket.
The problem is in what concepts are being considered "the same" when mapping two mostly-dissimilar things (a socket or pipe, vs a file).
poll(), select() et al don't define "readable" as meaning "a byte of data is available". They define it as "read() will not block".
When you read to the end of a file, read() returns "". This is the same as when you try to read from a closed socket. Conceptually, they are linking the concepts of "EOF" and "closed", not the concepts of "EOF" and "waiting for data". And indeed, if you call poll() on a socket that has been closed on the remote end, you will find it is "readable".
Ultimately, regular files just were never intended to be used as pipes. The abstractions just weren't chosen to work in that way.
> poll(), select() et al don't define "readable" as meaning "a byte of data is available". They define it as "read() will not block".
If that were true, poll() wouldn't return read-ready on a file-based fd until the data was actually sitting in buffers. After all, that's the only way to guarantee it won't block. That would actually be a useful semantic.
But what actually happens is that poll() on a file-based fd is basically a no-op that always returns true immediately, AFAIK. Someone could pull the disk drive cable before you actually call read(), in which case read() will indeed block, forever.
The existing semantic is useless. What argument is there in favor of a useless semantic?
You mean like in multiterm or something? I guess so. In emacs there are always many ways to do things, which is awesome.
You can also open the file and engage auto-revert-mode or auto-revert-mode-tail. In my status bar it shows up as ARev mode.
Also you can bind those to your auto-mode-alist so all .log files get the tail treatment or whatever.
Scrolling can get weird.
Theres some google-able settings for the timeout to revert, default is about a sip of water (a few seconds?) but I guess you could crank it down to 1 second or something.
I like doing this in emacs so its all in the same ecosystem, if I need to cut and paste something weird to test it or put the message into code or even more likely to cut and paste actual error messages into docs or comments, I can do it inside emacs.
It looks like from ~24.4 onwards there's support for various notification event sources for auto-revert[-tail]-mode, as determined by the file-notify--library variable. I don't appear to have anything compiled into my prebuilt OSX emacs though, time to update the ol' source tree and see if I can remember how to build it.
There's also lnav (http://lnav.org), the Logfile Navigator. It can do what 'tail -f' and 'less' do plus: syntax coloring, filtering, loading compressed files, interleaves log messages from multiple files, and more.
Many of the comments I see on here about using '-F' with tail instead of '-f', live searching, and so on are handled in lnav by default. I would really encourage folks to give it a try.
210 comments
[ 4.3 ms ] story [ 305 ms ] threadPersonally, I'd love to see some hot young talent just do a 100% redo of the standard gnu utils from an interface perspective. Just go crazy with new interface and display ideas, novel presentation modes, novel navigation, etc while still maintaining backwards compatibility. I could see a big disruption here.
For example: tail -F somefile.log|ccze -A
For your shell, there's a lot of funky colorful interactive customizations you can get on zsh and fishfish.
Of course, I'm not opposed to colours, I'm opposed to colours that look tacky / l33t hax0r, which would be most terminal stuff. I like 𝐛𝐨𝐥𝐝 better than trying to find colours I won't hate.
The use case that drove its development was needing to keep track of UUIDs across multiple logs - and grep --color will colorize its matches, but not differentiate between ones that have different content. With this, I could watch both logs as data was passed from one to the other and keep track of e.g. the orange one.
I also thought it would be nice to be able to use patterns from logstash's grok, so I wrote grokpat[2] to find patterns for me. A lot of grok's patterns use atomic groups, which aren't available in Python 2, so I wrote redi[3] to convert them from grok's syntax to Python compatible syntax.
So I can now colorize logs easily as follows:
[1] https://github.com/cromo/synesthesia [2] https://github.com/cromo/grokpat [3] https://github.com/cromo/rediThe downside: less buffers and tail -f prints directly. On a slow printing log this can cause events to show up slowly, and can cause performance issues on a fast moving log. If you're piping through a script for processing, tail -f is still the best bet. If you need multiple files, multitail is probably better. less +F hits a quick+easy operational niche and is available almost everywhere (whereas multitail is not).
$ less ./somefile
(Edit: formatting)I typically read logfiles with less +F, but it would be nice to create an alias for patterns in specific types of logs, so I don't have to remember them or rely on command history.
* Note that I needed to "quote" (^V^M) that carriage-return to get the search pattern to work.
Happy searching-and-following.
From creating vi (nvi, on NetBSD) macros, I'm used to thinking in terms of competely replicating the keystrokes that one would do interactively... so I tried it both ways (with and without ^M). The way I published is the one that works. Why the ^M ? Because if you're working interactively the search-pattern isn't submitted until you press Enter. nvi(1) (what NetBSD (and Free and OpenBSD) uses) will accept "-c" "command" arguments which are similar to the less(1) "+"... so, you can:
$ vi -c 123 ./myfile
and start editing "./myfile" at line 123. Nice for edit/compile/edit/compile dance that might happen if you're developing software. Play with that (and try your imagination with other ideas).
Have fun, happy exploring.
[1] Everything i've used on Linux in the last few years, not the vanilla one on the Mac, but the one in Homebrew
I've commented a bit more on them (in the context of other RSS tools) here:
https://www.reddit.com/r/dredmorbius/comments/1udv6i/further...
Folkert's site: http://www.vanheusden.com/ http://www.vanheusden.com/multitail/ http://www.vanheusden.com/rsstail/
Though not quite applicable to logfiles, I've written a script to tail my Newsbeuter feed URLs with Multitail support, for a console / terminal-based RSS feed.
tail -f whatever.log | grep --color=auto -C99 exception
And there are other ways pausing/scrolling back, such as tmux's scroll mode.
Also, if you are using a proper ssh client, opening a second ssh connection to manage the reacting to the log files is habit for me at this point. The only time I'm using tail -F is after a new configuration deployment. Otherwise, I'm looking at archival data in ELK.
[1] https://www.gnu.org/software/screen/
Thank you for the suggestion tho. :)
And, since we're being pedantic, when I said “vast majority of the software world”, that includes the OpenBSD project:
“Sending in bug reports
If possible, use the sendbug(1) command to get the bug into our tracking system. Sendbug requires that your system can properly send Internet email. If you cannot use sendbug on a functional OpenBSD machine, please send your bug report to bugs@openbsd.org.
Perhaps what you are sending in is a feature request, not necessarily a bug. New features are accepted, especially with code that implements your suggested new feature.”
http://www.openbsd.org/report.html#bugtypes
I'm not being "pedantic", I just don't think there is anything to change.
> FreeBSD has this
OpenBSD ≠ FreeBSD, and that's fine.
Yes, this is mentioned in the post, but I feel it is so big it would have deserved large, bold letters.
With less, you have to keep track of the current position by timestamp or other unique message, and it's easy to lose your place when the output starts streaming in. With tail, you only need a moment to mark your spot and then it's visually distinct even as more message come in.
This trick doesn't work if your log file has a bunch of identical lines and you want to keep an eye on their rate, though.
Enter method:
Your method:But the original poster posted a useful tip, and is now getting aggressive downvotes and comments like "This has nothing to do with what is being discussed in this subthread." I think that's unwarranted.
That's why suggestions of either named-marks or back-searches aren't considered equally-attractive alternatives to marking the scrollback with a batch of <return>s.
yank 10 lines, open a new file, paste the 10 lines, save the file.
Is that possible? My Googling last week didn't find a good solution.
http://stackoverflow.com/questions/17908555/printing-with-se...
(Actually, there's probably more than one despite their really good duplication grooming, but this was the first hit.)
As a junior programmer I got chewed out by my IT department because I was examining a (production) log file in vi. The sysadmin told me I should use less (actually more---this was on Solaris in ~2001). To this day I only read log files with less, but I've never figured out his objection. Negatives to using vi I can imagine are:
- I might write to the log file. That seems like a real worry, although I could also say `vi -R` to prevent it.
- Starving the production system of memory. I'm pretty sure he expressed this concern. Any insight into whether it is legit? Is less actually any better?
Obviously you really should have a log shipping & aggregation service so you can read logs offline, etc, but not every project is large enough for that, nor every org organized enough. So for the sake of argument my premise is, "Assuming you want to read a log on production . . ."
Besides — vim really shouldn't be used on log files. It's editor, not viewer.
No, vim also only keeps part of the file loaded, however it will try to count how many lines the file has.
I think the two concerns I stated are probably legit, but this one is boring to me. Vim is nicer than less for reading files. I have more navigation commands. I can yank part of the file and save it somewhere else. I can pipe a range of lines through awk/etc. I can switch between files. I can split my screen. Etc. Some of these are probably available in less too (more than more(1) supported in 2001), but I doubt all, and I already know the commands in vim. I'm interested in not clobbering my logs and not crashing the server, but if you tell me vim is an editor not a viewer, I'll ask Why?
Btw re-reading my words I don't mean to sound combative. But the point of my original question was to understand. I've been cargo-culting "use less for logs" for 14 years already.
Frequently, frequently I am in the position that I need to manipulate an overly-verbose log file to condense out the information I want. I could spend a half hour concocting wizard-like shell invocations, OR I could do it interactively in five minutes with vi...
Sometimes I want a viewer to just be a viewer.
Speaking of which: are there any Linux/Unix viewers that do have generalized syntax highlighting support?
Beyond that, I would probably correct a junior employee as well. Even if there's nothing wrong with it, it's not the right tool for the job. When I first started I got 'yelled' at for checking to see if a machine was on the network using tracert instead of ping. It works, but it's not the right tool for the job.
I'm generally in the habit of using `view` to do read-only vim. `less` works as well.
[0] http://vimdoc.sourceforge.net/htmldoc/starting.html#view
I think the problem was that the memory and processor were already getting stomped on (thus the need to look at the logs) and vim tried to do a lot of fancy stuffs to get more info on the file as a whole.
1) for user: less chance (pun intended) to actually change the file when all you wanted was to read it
2) for sysadmin: if sysadmin sees "less somefile.log" in bash history, he knows the user just read the log. If he sees "vi somefile.log" then he doesn't know if the user has also changed the log file (maybe not even knowing it).
The assumption is that you deal with non-malicious users who just make mistakes (which is often the case).
You should download the log file and view it locally. You should never run ad hoc commands on a live production sytem.
And there is no reason that you should have edit privileges on the log file anyway.
http://seclists.org/fulldisclosure/2014/Nov/74
Now I mostly use vi.
Display only lines which match the pattern; lines which do not match the pattern are not displayed. If pattern is empty (if you type & immediately followed by ENTER), any filtering is turned off, and all lines are displayed. While filtering is in effect, an ampersand is displayed at the beginning of the prompt, as a reminder that some lines in the file may be hidden.
But really, it's 2015. Are you really using a terminal emulator without a buffer search? Are you not using screen or tmux, both of which can do these things natively, with no external tools? Why? And what is wrong with backgrounding the tail command, running your grep, and then foregrounding the tail command?
If tmux can do that, then I'd consider it reason to switch.
tail -f /var/log/x.log | grep foo -A 20 -B 20
Most of the time.
This is the reason I always share how I do it in hn! LOL
There was also a time when there was a utility called inotail, bridging that gap until tail proper was improved. I very much preferred it over regular tail.
poll(), select() et al don't define "readable" as meaning "a byte of data is available". They define it as "read() will not block".
When you read to the end of a file, read() returns "". This is the same as when you try to read from a closed socket. Conceptually, they are linking the concepts of "EOF" and "closed", not the concepts of "EOF" and "waiting for data". And indeed, if you call poll() on a socket that has been closed on the remote end, you will find it is "readable".
Ultimately, regular files just were never intended to be used as pipes. The abstractions just weren't chosen to work in that way.
If that were true, poll() wouldn't return read-ready on a file-based fd until the data was actually sitting in buffers. After all, that's the only way to guarantee it won't block. That would actually be a useful semantic.
But what actually happens is that poll() on a file-based fd is basically a no-op that always returns true immediately, AFAIK. Someone could pull the disk drive cable before you actually call read(), in which case read() will indeed block, forever.
The existing semantic is useless. What argument is there in favor of a useless semantic?
This semantic wouldn't solve the "tail -f" problem, but at least it would be useful: http://cr.yp.to/unix/asyncdisk.html
You can also open the file and engage auto-revert-mode or auto-revert-mode-tail. In my status bar it shows up as ARev mode.
Also you can bind those to your auto-mode-alist so all .log files get the tail treatment or whatever.
Scrolling can get weird.
Theres some google-able settings for the timeout to revert, default is about a sip of water (a few seconds?) but I guess you could crank it down to 1 second or something.
I like doing this in emacs so its all in the same ecosystem, if I need to cut and paste something weird to test it or put the message into code or even more likely to cut and paste actual error messages into docs or comments, I can do it inside emacs.
There is also the command `tailf` which is quite similar to 'tail -f'.
2 characters make so much difference :)
I saved you three more!
Maximum savings!
tail -f sys.log | grep NETDEV
But I would definitely give this a try, thanks for sharing.
/thefox
Many of the comments I see on here about using '-F' with tail instead of '-f', live searching, and so on are handled in lnav by default. I would really encourage folks to give it a try.