Reduce the size of your SVG

...so recently i was playing with SVG, and i wanted to embed an older version of this map of Europe into my html . Currently it is about 500 KB, but as far as i remember, it was even heavier. Pretty bad, if you want to inline it, and your visitors have to download it. So how did i reduce the size to one tenth of it?

gzip?
Yes, gzipped version of SVG exists, but not when you inline it, so its a no-go in my case.

delete unnecessary regions
I didn't need Greenland and Russia for example, so i just erased them from the source file.

delete unnecessary metadata
My file was scattered through with inkscape:xyz and some other editor's metadata, which might be useful when editing the file, but useless when i publish it on the web. They had to go.

group and style wisely
Don't repeat the same styles/transformations on many elements, use grouping instead (<g> element)

remove unnecessary transformations
A lot of my regions contained a very little translate transformation, which had no visible result at all, so i chucked them out.

compact path data
According to the SVG specs, you can reduce the size of your path data by removing redundant white spaces from around commands and coordinates. Works great!

simplify path data
Ok, this one you will not be able to do in your text editor. The idea is to reduce the number of coordinates required to describe a path. Even if it means you will have a less precise path, the result might be satisfying. To do this, you have to have Inkscape. Select an area, and you will see in the status bar the number of nodes it consists of. Now select path/simplify from the menu. If you are lucky, your number of nodes will be less, if you are not, it will actually be higher:) For me it worked best, when i set the simplification threshold (Edit/Preferences - Misc) from 0.0020 to 0.01. Yes, you will have visible deviation from the original path, but in my case it was acceptable. Also note, that under a certain number of nodes (for me it was around 80) it is not worth to run the simplification.

reduce coordinate precision
Ok, so i still had (and you will most likely have) a precision of four decimals in my path coordinates. What if i reduce it to 3/2/1/God forbid zero ?! Yep, no visible impact on the web, but huge saving on the filesize! This is the ultimate file size killer, if you can afford it.

NOTE: Other good tips you can read here and here.

Ext 2.2 is out, but where are the old releases? [UPDATE]

If you like Ext JS, the new 2.2 version brings out some nice features for you. No question Ext is a great library and worth to play with, despite the recent disputes around the license change from LGPL to GPL.
My only frustration with Ext was only a few days ago, when i was about to make modifications in an open source project that had an earlier Ext 2.0 bundled, but not fully! So where can i get the full Ext 2.0 package to add some adapter files that were missing?

1. Ext website? Forget it. On the download page you can only download the latest, on the "build your own" page you can have 1.1.1 and 2.1. No matter if you try to trick the forms, you will not get any 2.0 version :)
2. Second idea: SVN! Hehe, forget it. Ext JS does NOT have open SVN access (that is free).
3. Hmm, if you read the threads around the license dispute, you will eventually come accross a project that forked (basically just copied and published) the latest 2.0.2 LGPL version of Ext. It is called openEXT, a few days ago it was still available on sourceforge, by now it seems to have vanished. Anyway it was questioned if this fork was legal or not, so the question is still: where can we get a legal copy of 2.0?
   
4. My frustration grows. What next?
5. The idea pops out: lets look up some open source projects that use Ext, and have access to past releases. I am sure there are many, but the one i had immediate success with was SugarCRM. Download version 5.0.x, and there you go!

NOTE: I have no intention of ruining the business of ExtJS here, my only intention was to fix something that used an older version of Ext. Putting aside licensing questions, i strongly believe that closing down the access to older versions is rarely a good idea. It contradicts the philosophy of open source and the Internet itself.

UPDATE!

As Anonymous pointed out in the comments, the old releases are indeed available on the ExtJS website!
To my defense to reach it you have to go: home - support / community / learning center - API Reference and Demos(?) / Archived downloads of previous releases. Not the most logical you would say, and no link from the downloads page. Also if you ask Google who links to this page, you will get zero result. It doesnt appear in their robots.txt as disallowed, yet somehow it is not being indexed.

UPDATE2: OpenEXT seems to be back by now.

UPDATE3: There is a gcode project called extjs-public, which provides svn access to publicly released Ext JS versions. It seems to hold releases beggining of 2.0.2. Enjoy!

Javascript documentation tools

...so you were a good programmer and you have added javadoc style comments in front of your javascript functions/classes/methods? Good boy! Now lets see what you can do with them. Solutions in order from worst to best (just to keep you entertained):

Javascript Documentor (JSD)
It is java based what would be super useful in ant scripts, however it expects me to write comments like:

/// <name>MyFunction</name>
/// <param name="param1">The first parameter passed in.</param>
/// <param name="param2">The second parameter passed in.</param>

Whaaaat?
1. Programmers will NEVER write that much metadata
2. Javadoc syntax is already de-facto standard, why invent a new one?
3. Javadoc syntax is already supported by many IDEs/editors
Next, please!

jGrouseDoc
Well, first of all, have you guys ever seen your page in Firefox? You would be disappointed for sure, as it gives a big blank page. Yes, i know you were trying to be smart with some ajax content loading, it just didnt work.
Anyway, after switching to IE, and downloading the tool, it turns out that it is java based, it can nicely be integrated with ant (it even comes with a fairly complex build script), but the only thing it recognized in my javascript file was... the filename itself. Hmm, thats good for a start, but lets move to the next one.

NaturalDocs
It is PERL based. It is meant to be a documentor for multiple languages, javascript not being among the fully supported ones. Lets give it a try though. Issue command. Read manual. Issue command. Read manual. Ah, ok. Issue command. Generation done. Hmm, plenty of files and directories, but not a single html. Apparently it couldnt digest my file. Next one!

JSDoc
Not to be mistaken with the JSD above! Because this is good:) It is PERL based, and will be happily running after you have downloaded the HTML-Template package. It magically recognized all my kinky objects and methods, and generated a classical jsdoc for them. Well done, i will stick with this one.

JSDoc Toolkit [UPDATE]
Just to confuse you more, there is this "toolkit" version. It seems to be the younger brother of the above mentioned JSDoc, but it is completely javascript based. Getting rid of the PERL dependency is great but it comes at a price. The javascript based processing is terribly slow, you have to see it for yourself if it is worth to switch. If you decided to switch, you can find a nice template here.

NOTE: On this wikipage you will find a bunch of other documentors, some of them are general, a few, apart from the above mentioned, are specific for javascript.

Happy documenting!

UPDATE: jGrouse website is now functional.

Online OCR

I was thinking recently how to speed up in an intelligent way the creation of image maps with my online utility. Often users try to map images like this:

The user will select "Home" and try to map it, giving it an url, adding an alt attribute. Now what if we could prefill the alt attribute with the text "Home"? Optical character recognition (OCR) in javascript? Wow, that would rock!
Well, technically we can access pixel data with the advent of the canvas element, but since OCR is a processor heavy operation, as i see today (better to say as Google sees it), noone wasted the time to write a javascript based OCR yet. :)
Now what if we use some server-backed online service?
Technically speaking again, it is possible to crop one part of the image, get the image data, post it to the online OCR, and parse the result. Unfortunately these online services didnt produce too good results with the test image above. I know, it is noisy jpg and too small, but still, it is a real life example.

Anyway, here comes the list of online OCR services that i found, in order from best to worst, result of the process in brackets:

• OCRopus (Hume bxhuuzus \ sewn;)
• WeOCR (Wom_ | Abo__ __ | ____|___)
• Startek OCR demo (?0??. ?0????5. ????.??5)

The slow echo

...so i was about to profile a web application written in php, and as all of us does when doing so, i scattered the code around echoing microtime differences. (Ok, ok, some of you might use log instead.)
To my greatest surprise the part where the application showed significant slowdown in some cases, was a simple echo statement. Well, i wont be able to optimize that, will i? Why can this be happening?
After a bit of googling, i came across this site, where they suggest to split up the echo to smaller chunks to avoid network fragmentation. Hmm, i gave it a try, but no difference. It is a rather old post so i was scrolling down on the comments (most of them are engaging into the question of determining MTU and completely miss the point) to see if people still face this problem, and what are their solutions. The very last post (at that time) suggested simply using output buffering.

Well, my application was already using ob, but as i inspected the code more closely, one of the modules that was included early in the page generation, simply switched ob off!
Damm, switching it back speeded up the application again!

So what is happening in the background?
I am not sure about the implementation of echo, but i guess when echoing a large string over a slow network, if output buffering is not used, echo waits until it has sent the last byte. In other words the network latency comes down to php level.
However, if you use output buffering, you can generate the whole page quickly, then have the webserver send it to the client over the network, all the latency is offloaded from php and will not reflect in the execution time.

To sum up, output buffering is your true friend.

SugarCRM redundant indexes

...ok, so i was doing some database tuning recently to speed up one of our deployed instance of SugarCRM, when i became alert of an interesting phenomenon.

The table accounts not only had a primary key(id), which is normal, but also had a composite key(id, deleted) named idx_accnt_id_del. Now wait a minute! Even if you perform a query with id AND deleted criteria, id already points to a single record in the table, since it is a unique primary key. Thus the composite key makes absolutely no sense!
Mysqlperformanceblog, which i have read a lot lately, confirms my findings.
I didn't try any of the tools they mention in the comments section, but by quickly searching for "id_del" in Sugar vardef files reveals that the following tables have the same problem:
- roles
- acl_actions
- acl_roles
- fields_meta_data

These tables are usually small, and don't have frequent insert operations, so it is not a big performance issue, but the aforementioned accounts can grow big, and managing an unnecessary index with a length of 38 bytes is just a burden on the database. The redundant index is present in an older Sugar 4.5 and the latest 5.0.0f, too.

UPDATE: I posted it on Sugar forums, lets see the feedback:)

Web applications going offline

...so you still remember the times when Internet was loud of desktop applications going online, huh? Email was one of the earliest obvious implementations, slowly followed by the more difficult applications: online document, spreadsheet, presentation editing, agenda, todo list, contact management, image editing, imagemap editing ;), etc. And you started to use one or more of these online wonders, and you were happy. Obviously sooner or later you realized you are pretty much stuck when you are offline. Application developers also had this in mind, and now in 2008 we have a few handsome tools to transform some of these applications to work in offline mode.

The trend might seem ironic, as applications that were desperate to go online now try to crawl back to your desktop, but in reality this only means progression. We benefit hugely of the interaction of the web and the desktop. Myself played some hours with Adobe AIR (Gears and BrowserPlus still waiting to be discovered but they also look great), and as i see most applications can gain a lot with options like drag and drop, file operations, local storage.

Currently Gears and Air offers rather different approaches to offline working. Which one will be the winner? Only future can tell, both seem very strong at the moment.

LOLCODE

HAI! Seriously, did you know there is a programming language called LOLCODE? I did not, but apparently it is very popular, among the implementations we can even find PHP and Javascript parsers.

Hello Hai World Example:

HAI
CAN HAS STDIO?
VISIBLE "HAI WORLD!"
KTHXBYE

Firefox 3 changes in file input

...so try to access a file input's value property in FFX3. What do you get? The filename. What do you get in IE and FFX2? The filename AND the path.
Well, if your script has been relying on this behaviour, you might find yourself in trouble. So far the only solution i found hides in the comments section of this post (great showcase of new functionality on the other hand btw).

To save you the time, you have to do the following:
- go to about:config page
- swear that you wont make any trouble :)
- set signed.applets.codebase_principal_support to true (search for "applet" in the quick search box)

And in your script something like this:

<input name="uploadfile" onchange="
if (document.myform.uploadfile.files) {
//ffx3 - try to have access to full path
try {
netscape.security.PrivilegeManager.enablePrivilege( 'UniversalFileRead' )
}
catch (err) {
//need to set signed.applets.codebase_principal_support to true
}
};
document.myform.file_path.value = document.myform.uploadfile.value" type="file">
<input name="file_path" type="xhidden">

Now whenever you will try to run the onchange script, FFX3 will display a dialog window where you can confirm access to the full file path.
(Sorry about the formatting i am still examining whats the best way to post code in blogger)

UPDATE: Another approach can be found here.

PHP RSS parsers

...so the other day i figured out that the parser i used so far (LastRss) was not able to read Atom feeds. Since our beloved Blogger only publishes atom feeds i needed to find a cure for the problem.
One easy solution is to ask FeedBurner to convert between rss and atom, which works pretty well, but better not mess with the Gods, lets do it the proper way.
Then i came across this blog entry, that compared some of the existing solutions. Finally i chose SimplePie, which works brilliantly ever since. To have an overview of the solutions, here is my little comparison:

LastRss
+ simple
+ fast
+ small footprint
- no support for Atom
- website looks interesting recently (says: It works!)

Zend_Feed
+ part of Zend Framework
+ supports Rss and Atom
+ well documented
- supports PHP5 only

Magpie
+ supports Rss and Atom ("with few exceptions" :))
+ long time out there
- long time out there :) seems like it has never really grown up

SimplePie
+ seems like it supports everything
+ well documented
+ fresh and stable project
- one heavy includefile if it bothers you (350K)

XML_RSS (Pear)
- i am not sure, but i think only supports Rss feeds
- Pear dependencies
- "Oldest open bug: 326 days"

RSS_PHP
- supports Rss only
- PHP 5 only (i know, i know, we should all live in a PHP5 world, but we dont)
- "A commercial version (v3) of the RSS Parser / XML Parser for PHP [rss_php] is now released and available for download. This release fixes a couple of tiny bugs and adds far more functionality...Our original version (RSS_PHP v1) is still freely available." - so you can get a buggy version for free, the commercial version costs 15USD.

Evolution of Gmail chat

Just a short overview of how Gmail chat evolved since its first appearance. And it has to be said it works very well. However i started to be a fan of Google Talk Labs edition. I dont know since when it has this feature, but i just love the fetching of youtube vids and picasa albums straight in the chat window so i dont have to visit the site itself, very well done!

Long time no news

Hi all, sorry for being silent for a while, but in the past weeks i had a lot of things to do. From one part i have moved to a new apartment where is still no Internet installed, from the other hand i was on holidays. As a result my feed reader contains around 700 unread items i should browse through when i find a little free time in the office:)
Looking back the most important thing that happened during the time is the release of the new Firefox.
To have an overview what other browsers are doing quirksblog made a writeup.

My personal experiences so far w FFX 3
- better memory usage - i can run the browser for hours, still get a memory usage of around a 100 and sth Mb.
- faster user experience, faster tab switching
- all my extensions work, EXCEPT firebug ...ah i would need that so much!!!
- my own extensions work, i only need to change maxversion number
- the magicbar, or whatever it is called, i dont really like, i like the logic of ordering, but i think its hard to see the url, and that is what usually important for us ...maybe i will get used to it

Happy browsing everyone!

UPDATE:
- firebug seems to have a beta version that works with FFX3, so far i experienced no problems with it
- Google browser sync seems to be discontinued. I will have to search an alternative solution to sync bookmarks.

UPDATE2:
- Foxmarks is an excellent replacement for google browser sync so far
- although the memory footprint of FFX3 is much better, i often find myself in situations when the processor usage is unreasonably high (around 50%). I have to restart my browser to stop burning my CPU. No good.
- i am getting used to wonderbra wonderbar :)