Bandwidth problem? investigate your log files

On some of my APEX sites I seem to have a bandwidth problem. Not that the sites become slow, they just eat so much bandwidth without reason.


We set a limit on the bandwidth a site can take a month, just so we are in control about the hosting/bandwidth fees we need to pay. If for some reason we see the site is eating a lot of bandwidth (which we didn't expect) we enable full logging for that site.

Next we investigate the logs. You can read the raw data, but you can compare that with reading a tracefile. You can understand things if you look closely at it, but it's not very user friendly, so for tracefiles you use tkprof to get a more human readable output. Same counts for the Apache logs, there are programs that can format that in a better way. I used AWStats to convert my logs to a nice overview. The result looks very much like statcounter stats (without the chart).


Out of these logs you can get very valuable information about your customers, not only do you see how many people you have per month/day/hour or which IP is accessing your site, you also get statistics about the platform they use.


For me it was interesting to see that for that site Windows and IE are less used than what I expected.

Further on in the log files you see how people reached your site; which keywords they used in a search engine or from which other site they came and how long they stayed on your site.

For me most important was to get an insight where my bandwidth went to. It didn't take me long to see that Robots/Spiders where the problem! It's unbelievable that Yahoo's bot is eating so much bandwidth!


To solve this issue you need to create a robots.txt with some parameters like specified in this link.

Looking at your stats and investigating the full logs can be very useful from time to time...

Supported by hosting

Update: I love PLSQL and ... (.net)

I got some messages about the title of my previous blog post and the picture you get when clicking on Steven Feuerstein's photo on the iloveplsqland.net site.

The title of my previous blog post was a bit to "shock" some people and see what it would do to the stats of my site. ;-) Nothing spectacular of course... but I got a lot of mails or comments about this title. You know now why I did it... it's not Steven's opinion he likes PLSQL and .NET, of course NOT ;-)

To avoid problems or comments in the future I changed the title to a normal one you could expect.

The website got in the meanwhile some small updates. Nothing spectacular, just some improvements Bryn asked.
For ex.: "Can we get an estimate of the amount of PL/SQL they have (both no. of compilation units and no. of source code lines)."
Implementation: On the site you'll see two new fields: Estimated PL/SQL code volume: Lines [ ] Programs [ ]


And finally about the picture... that's Steven (when he was a bit younger)!

I love PLSQL and...

John Scott and I are very pleased to announce a website we created for our friend, Steven Feuerstein.

A while ago I blogged about "Apex Evangelist meets PLSQL Evangelists" (see also this pic). I met Steven for the first time in real life at ODTUG last year. I've always been admiring him... the way he presents, his charisma, friendliness and of course his PLSQL knowledge.
So, doing this project was a real honour!

Some blogs already announced (even before it was "real" production) Steven Feuerstein's new website, called "I love PLSQL and ... (.net)". The url some websites referenced was wrong. This is the url you should use to see the site: http://www.iloveplsqland.net

Naturally the site was fully created in Oracle Application Express ;-)
It's a fairly simple site, but if you look good, you'll see the integration of some 3rd party products like "Highslide" (click on the picture of Steven!) and "Captcha" (register for an account). You'll also see some AJAX (Asynchronous Update) in the Flash Charts.

We hope you'll like the site and hope it can be a step towards the improvement of the PLSQL language.

John and I really enjoyed making this site... Shall we also make such a site for APEX?

Update: I first wanted to shock people with the title "I love PLSQL and .NET", but finally decided to change it again as I got a lot of reactions ;-) ... (see also other blog post)