urlclassifier3.sqlite woes on Firefox 3
June 27, 2008
Ever since I’ve started using Firefox 3, I’ve been noticing that the ‘profile folder’ (where FF stores all the user’s personal info, extensions info etc) was steadily growing in size. Right from my initial usage of Firefox, I’ve not been using the default folder path for this profile but using the firefox -p startup option, create a Firefox_Profile folder at a different location so that the personalization could be carried over from version to version.
With Firefox 3, this profile folder started to bloat in size very often. Whilst with FF2 it was almost always nearabout 50 megs, with FF3 it grows to ~100 MB after a few hours of browsing. I looked into the individual files and bam! the culprit stood out to be a urlclassifier3.sqlite file which was topping out at 68 MB 
. Alarmed at this, I Googled around and found this Mozilla Knowledge Base article on the same.
The urlclassifier2.sqlite file contains information on known phishing sites that is used to warn you should you visit one. The information is downloaded by default from a Google database once a day. This functionality can be modified or turned off in “Tools → Options → Security
Firefox 3 contacts Google at each startup to get an updated list of known phishing websites and updates the same in the urlclassifier3.sqlite file. Agreed this is what makes our browser safe and secure but why is it bloating up in size? Are the phishing sites on the web increasing as fast as the size of this folder? Naah! Apparently, there’s a known bug in FF specific to this urlclassifier file which says that it has a lot of vacuum/blank space within and there ought to be a mechanism to automatically compress it effectively.Whatever might be the cause, its rapid size increase is somewhat disconcerting. Added to it is the fact that upon each startup, FF contacts the Google servers to get an updated list, which might be a drain on the CPU resources and in turn the notebook batteries (if you’re using one i.e.). There have been discussions about it over at the Mozillazine Forums in this thread and this thread and there seem to be a lot of people disconcerted by it.
The Solution? Well, the knowledge bas article suggested turning off the Phishing options in FF, which would mean unchecking of the following options under Tools>Options>Security:
Though that would mean compromising on the security aspect. But its alright of one isn’t too prone to making transactions over the web and visiting out-of-the-way websites. After unchecking the options, delete the urlclassifier3.sqlite file from the profile folder and it will never bother you again. A 0KB file may be created at the next firefox start but it won’t bloat in size as previously. I hope the bloating problem is fixed soon and we can have a very safe and secure browsing experience…minus the size increases etc. 
[Disclaimer: Delete the urlclassifier3.sqlite file from your firefox profile at your own risk. Please do not hold me responsible for anything untoward that might happen to your browser when you do that!]
Entry Filed under: Awareness, Daily Rants, Firefox, Firefox 3, Info, Tech, Web Travails. Tags: Anti-Phishing, Bloatware, CPU Usage, Database, FF3, Firefox Bug, Firefox Profile, Firefox Woes, Google, Mozilla, Mozilla Knowledge Base, Mozillazine, Phishing, sqlite, urlclassifier, urlclassifier.sqlite, urlclassifier2.sqlite, urlclassifier3.sqlite.
30 Comments Add your own
Leave a Comment
Some HTML allowed:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <pre> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>
Trackback this post | Subscribe to the comments via RSS Feed
1.
wilbur smith | January 30, 2009 at 00:36
I’ve looked at the contents of the file, 75% of the file is filled with zeroes. So, obviously, much of the problem derives from the fact the the Mozilla guys chose a really really crappy database system to store the info.
2.
x | March 31, 2009 at 19:22
It’s not a “crappy database system” (it’s SQLite), it’s that they half-assed the implementation. They don’t bother doing housekeeping on the file, nor do they compact it.
3.
Ralph | July 9, 2009 at 23:55
1) Wouldn’t this “phishing site” database (urlclassifier.sqllite) best be kept globally, in the firefox install directory?
If SQLlite is used for multiple purposes
2) @x, who is ‘they’ The Firefox developers? Can I compact the database myself with an SQLlite utility?
Is there a bug report submitted for them to fix?
Why would a database file need to be periodically compacted, when the software developers should be doing that automatically?
I have a lot (thousands) of bookmarks, and one of my places.sqllite is hopelessly large, hosing the browser. If FF doesn’t compact the database, what’s the recommended method for cleanup?
4.
Richard | November 4, 2009 at 19:55
Looking at the description it has to be modified by the end user. On systems that enforce security of the application directory it would not be possible to place it there.
On a Linux system it could be possible to place it in a central location (traditionally somewhere in /var) but then something like a scheduled task will have to go off to maintain it. On a properly managed multi-user Linux or UNIX system this would make sense.
It seems good for privacy that the checking is done locally, rather than asking a central server every time you visit a site. The latter would not require everyone to keep a copy of the database, but would introduce other problems.
You can ask sqllite to vacuum the file. It reduced it from 55M to 44M so not much.
5. Firefox Directory in RAM « Necoro’s Blog | May 8, 2009 at 14:51
[...] for the phishing protection (i.e. ask google for each site you are visiting :/). See for example this post about the details (or use [...]
6.
L | July 27, 2009 at 17:51
I was getting emails from the IT department at my work saying I was using too much disk space. I couldn’t understand how so went looking for large files. I realised it was the above file and shocked to learn it had grown to 922 GB (Yes GB not MB)! I’ve deleted it so my employers should be happier now!
7.
Chittaranjan | July 27, 2009 at 22:18
8.
Hastabum | September 27, 2009 at 05:27
922 GB is not that much. My office computer’s urlclassifier3 file grew to 219 TB (yes that’s terabyte, not MB, not GB… TERABYTE).
I needed over 400 external hard drives just to store that file! My boss wasn’t very happy about it.
9.
Chris Daniel | August 25, 2009 at 05:01
The growth of this file (especially if it is mostly zero-filled) is probably due to a lack of vacuuming. See http://sqlite.org/lang_vacuum.html for more info on vacuuming a database.
You can vacuum the database by using an sqlite tool, e.g., on Linux, “sqlite3 ‘VACUUM; REINDEX;’” or you can try this tip from Lifehacker (which doesn’t require a restart): http://lifehacker.com/5344418/make-firefox-faster-by-vacuuming-your-database
10.
Chittaranjan | September 27, 2009 at 15:24
Jeez! I am surprised the HDD even had that much of space