Friday, September 7, 2012

Filtering AppEngine dev_appserver logs on unix bash


Just a small post to make a note of something effective that I use. While working with the appengine, one often has to wade through lots of log statements, and having a lot of log statements is a good thing in my opinion. However, one can miss crucial information when the output isn’t very readable. I use the following command to filter and also color the output on bash shell on my Ubuntu machine.

dev_appserver.py . 2>&1 | grep --line-buffered -v -e 'GET /static' -e 'POST /stats' -e 'GET /favicon.ico' 2>&1 | egrep --color=always '^|WARNING.*|ERROR.*'

A little explanation for those who might find it helpful …
dev_appserver.py .: I prefer to run the app from within its directory.
2>&1: appengine puts out the log on stderr, and I’m redirecting it to stdout.
--line-buffered: because otherwise it won’t flush the output to the screen for every line.
grep -v: to show lines except that matching the lines that I don’t want.
WARNING.*: so that it colors the line up to the very end.

Creating an alias

Now, I surely wasn’t going to type all of that out each time. So I created an alias for it. For my bash shell, I added a line in ~/.bashrc as below:

alias da=”dev_appserver.py . 2>&1 | grep --line-buffered -v -e 'GET /static' -e 'POST /stats' -e 'GET /favicon.ico' 2>&1 | egrep --color=always '^|WARNING.*|ERROR.*'”

That worked well, until I wanted to add parameters to the alias, so that I could occasionally do a --clear_data to start with a fresh datastore. However bash does not support parameters to alias. So currently, this is what I have in my .bashrc.

myda() {
dev_appserver.py $1 $2 2>&1 | grep --line-buffered -v -e 'GET /static' -e 'POST /stats' -e 'GET /favicon.ico' 2>&1 | egrep --color=always '^|WARNING.*|ERROR.*'
}
alias da=myda

So now I can do this on the command line:
da .: to start the dev_appserver in the current directory.
da --clear_data .: to clear the datastore and start the dev_appserver in the current directory.

Hope that is useful for you!


4 comments:

  1. I stood motionless for long years with windows, as3 and PHP. But I still want to progress. Like this song: too old to rock and roll but too young to die. Is possible colorize log mesages? Again: congratulations for the initiative. it's a great incentive for beginners

    ReplyDelete
  2. Great and simple post you shared. Thanks for pointing that in your post.

    http://www.digitalbrief.com

    ReplyDelete
  3. Thanks for your article! I have been looking for quite a long time and fortunately I read this article! I wish you would continue to have valuable articles like this or more to share with everyone!
    abcya

    ReplyDelete

If you think others also will find these tutorials useful, kindly "+1" it above and mention the link in your own blogs, responses, and entries on the net so that others also may reach here. Thank you.