2008-07-29

Online OCR

I was thinking recently how to speed up in an intelligent way the creation of image maps with my online utility. Often users try to map images like this:

The user will select "Home" and try to map it, giving it an url, adding an alt attribute. Now what if we could prefill the alt attribute with the text "Home"? Optical character recognition (OCR) in javascript? Wow, that would rock!
Well, technically we can access pixel data with the advent of the canvas element, but since OCR is a processor heavy operation, as i see today (better to say as Google sees it), noone wasted the time to write a javascript based OCR yet. :)
Now what if we use some server-backed online service?
Technically speaking again, it is possible to crop one part of the image, get the image data, post it to the online OCR, and parse the result. Unfortunately these online services didnt produce too good results with the test image above. I know, it is noisy jpg and too small, but still, it is a real life example.

Anyway, here comes the list of online OCR services that i found, in order from best to worst, result of the process in brackets:

2008-07-28

The slow echo

...so i was about to profile a web application written in php, and as all of us does when doing so, i scattered the code around echoing microtime differences. (Ok, ok, some of you might use log instead.)
To my greatest surprise the part where the application showed significant slowdown in some cases, was a simple echo statement. Well, i wont be able to optimize that, will i? Why can this be happening?
After a bit of googling, i came across this site, where they suggest to split up the echo to smaller chunks to avoid network fragmentation. Hmm, i gave it a try, but no difference. It is a rather old post so i was scrolling down on the comments (most of them are engaging into the question of determining MTU and completely miss the point) to see if people still face this problem, and what are their solutions. The very last post (at that time) suggested simply using output buffering.

Well, my application was already using ob, but as i inspected the code more closely, one of the modules that was included early in the page generation, simply switched ob off!
Damm, switching it back speeded up the application again!

So what is happening in the background?
I am not sure about the implementation of echo, but i guess when echoing a large string over a slow network, if output buffering is not used, echo waits until it has sent the last byte. In other words the network latency comes down to php level.
However, if you use output buffering, you can generate the whole page quickly, then have the webserver send it to the client over the network, all the latency is offloaded from php and will not reflect in the execution time.

To sum up, output buffering is your true friend.

2008-07-25

SugarCRM redundant indexes

...ok, so i was doing some database tuning recently to speed up one of our deployed instance of SugarCRM, when i became alert of an interesting phenomenon.

The table accounts not only had a primary key(id), which is normal, but also had a composite key(id, deleted) named idx_accnt_id_del. Now wait a minute! Even if you perform a query with id AND deleted criteria, id already points to a single record in the table, since it is a unique primary key. Thus the composite key makes absolutely no sense!
Mysqlperformanceblog, which i have read a lot lately, confirms my findings.
I didn't try any of the tools they mention in the comments section, but by quickly searching for "id_del" in Sugar vardef files reveals that the following tables have the same problem:
- roles
- acl_actions
- acl_roles
- fields_meta_data

These tables are usually small, and don't have frequent insert operations, so it is not a big performance issue, but the aforementioned accounts can grow big, and managing an unnecessary index with a length of 38 bytes is just a burden on the database. The redundant index is present in an older Sugar 4.5 and the latest 5.0.0f, too.

UPDATE: I posted it on Sugar forums, lets see the feedback:)

2008-07-19

Web applications going offline

...so you still remember the times when Internet was loud of desktop applications going online, huh? Email was one of the earliest obvious implementations, slowly followed by the more difficult applications: online document, spreadsheet, presentation editing, agenda, todo list, contact management, image editing, imagemap editing ;), etc. And you started to use one or more of these online wonders, and you were happy. Obviously sooner or later you realized you are pretty much stuck when you are offline. Application developers also had this in mind, and now in 2008 we have a few handsome tools to transform some of these applications to work in offline mode.

The trend might seem ironic, as applications that were desperate to go online now try to crawl back to your desktop, but in reality this only means progression. We benefit hugely of the interaction of the web and the desktop. Myself played some hours with Adobe AIR (Gears and BrowserPlus still waiting to be discovered but they also look great), and as i see most applications can gain a lot with options like drag and drop, file operations, local storage.

Currently Gears and Air offers rather different approaches to offline working. Which one will be the winner? Only future can tell, both seem very strong at the moment.

2008-07-09

LOLCODE

HAI! Seriously, did you know there is a programming language called LOLCODE? I did not, but apparently it is very popular, among the implementations we can even find PHP and Javascript parsers.

Hello Hai World Example:
HAI
CAN HAS STDIO?
VISIBLE "HAI WORLD!"
KTHXBYE

2008-07-03

Firefox 3 changes in file input

...so try to access a file input's value property in FFX3. What do you get? The filename. What do you get in IE and FFX2? The filename AND the path.
Well, if your script has been relying on this behaviour, you might find yourself in trouble. So far the only solution i found hides in the comments section of this post (great showcase of new functionality on the other hand btw).

To save you the time, you have to do the following:
- go to about:config page
- swear that you wont make any trouble :)
- set signed.applets.codebase_principal_support to true (search for "applet" in the quick search box)

And in your script something like this:

<input name="uploadfile" onchange="
if (document.myform.uploadfile.files) {
//ffx3 - try to have access to full path
try {
netscape.security.PrivilegeManager.enablePrivilege( 'UniversalFileRead' )
}
catch (err) {
//need to set signed.applets.codebase_principal_support to true
}
};
document.myform.file_path.value = document.myform.uploadfile.value" type="file">
<input name="file_path" type="xhidden">


Now whenever you will try to run the onchange script, FFX3 will display a dialog window where you can confirm access to the full file path.
(Sorry about the formatting i am still examining whats the best way to post code in blogger)

UPDATE: Another approach can be found here.

2008-07-02

PHP RSS parsers

...so the other day i figured out that the parser i used so far (LastRss) was not able to read Atom feeds. Since our beloved Blogger only publishes atom feeds i needed to find a cure for the problem.
One easy solution is to ask FeedBurner to convert between rss and atom, which works pretty well, but better not mess with the Gods, lets do it the proper way.
Then i came across this blog entry, that compared some of the existing solutions. Finally i chose SimplePie, which works brilliantly ever since. To have an overview of the solutions, here is my little comparison:

LastRss
+ simple
+ fast
+ small footprint
- no support for Atom
- website looks interesting recently (says: It works!)

Zend_Feed
+ part of Zend Framework
+ supports Rss and Atom
+ well documented
- supports PHP5 only

Magpie
+ supports Rss and Atom ("with few exceptions" :))
+ long time out there
- long time out there :) seems like it has never really grown up

SimplePie
+ seems like it supports everything
+ well documented
+ fresh and stable project
- one heavy includefile if it bothers you (350K)

XML_RSS (Pear)
- i am not sure, but i think only supports Rss feeds
- Pear dependencies
- "Oldest open bug: 326 days"

RSS_PHP
- supports Rss only
- PHP 5 only (i know, i know, we should all live in a PHP5 world, but we dont)
- "A commercial version (v3) of the RSS Parser / XML Parser for PHP [rss_php] is now released and available for download. This release fixes a couple of tiny bugs and adds far more functionality...Our original version (RSS_PHP v1) is still freely available." - so you can get a buggy version for free, the commercial version costs 15USD.

2008-07-01

Evolution of Gmail chat

Just a short overview of how Gmail chat evolved since its first appearance. And it has to be said it works very well. However i started to be a fan of Google Talk Labs edition. I dont know since when it has this feature, but i just love the fetching of youtube vids and picasa albums straight in the chat window so i dont have to visit the site itself, very well done!

2008-06-20

Long time no news

Hi all, sorry for being silent for a while, but in the past weeks i had a lot of things to do. From one part i have moved to a new apartment where is still no Internet installed, from the other hand i was on holidays. As a result my feed reader contains around 700 unread items i should browse through when i find a little free time in the office:)
Looking back the most important thing that happened during the time is the release of the new Firefox.
To have an overview what other browsers are doing quirksblog made a writeup.

My personal experiences so far w FFX 3
- better memory usage - i can run the browser for hours, still get a memory usage of around a 100 and sth Mb.
- faster user experience, faster tab switching
- all my extensions work, EXCEPT firebug ...ah i would need that so much!!!
- my own extensions work, i only need to change maxversion number
- the magicbar, or whatever it is called, i dont really like, i like the logic of ordering, but i think its hard to see the url, and that is what usually important for us ...maybe i will get used to it

Happy browsing everyone!

UPDATE:
- firebug seems to have a beta version that works with FFX3, so far i experienced no problems with it
- Google browser sync seems to be discontinued. I will have to search an alternative solution to sync bookmarks.

UPDATE2:
- Foxmarks is an excellent replacement for google browser sync so far
- although the memory footprint of FFX3 is much better, i often find myself in situations when the processor usage is unreasonably high (around 50%). I have to restart my browser to stop burning my CPU. No good.
- i am getting used to wonderbra wonderbar :)

2008-05-09

SugarCRM Event in Portugal

Just one week from now we will have a big SugarCRM event in Portugal. It is organized by DRI, amongst the speakers we will have Clint Oram, vice-president and co-founder of SugarCRM. I am really looking forward to be there, and i am also proud that i took big part in two of the projects that will be presented as case studies.

Website of the event (portuguese only): www.eventocrm.com

2008-05-07

Firefox + Blogger crash

For some reason my previous post keeps crashing my FFX2. Maybe the list is way too long:) The markup of the post itself seems like valid HTML to me, so it must be something else. My plugins, or other markup of blogger? It's a mistery, if it happens with you, too, let me know and sorry about the inconveniences.

PHP Certification Study Guide Errors

If you have been already preparing for Zend PHP5 Certification exam from the official study guide, you know it has many annoying errors and misleading statements. Up until the beginning of April there was no forum/listing where you could cross-check the errors, but now we have this errata in our hands. Its pretty complete, i only missed two thing that i noticed myself while i was studying. I added these in the comments section of the page. For the sake of completeness i repeat the list hereby. Happy studying!

First Edition Errata

Page 7:
reads "If you need to output data through a function, you can use print() instead". Person reporting comments that "PHP manual makes it clear that print is not really a function but rather a language construct" So perhaps we just need to change the wording.

Page 11:
reads "When converted to a number or a STRING, a boolean becomes ... 0 otherwise". Compare this to
http://www.php.net/manual/en/language.types.string.php
saying:
Boolean FALSE is converted to "" (the empty string).

Page 12:
top line reads "...want to convert to enclosed in brackets"... this should read "want to convert to enclosed in parentheses"

Page 17:
should read $a = int('Test'); //$a==0 ... echo ++$a; //outputs 1. Otherwise the output will read "Tesu" because of the incrementing

Page 19:
lines 2,3: If both bits are either set or unset, the resulting bit is unset

Page 26:
check versus table in the manual http://www.php.net/manual/en/language.operators.php

Page 29:
on while loops: should say "they allow you to perform a series of operations until a condition evaluates to false"

Page 34:
the line "if($oldErrorHandler) will cause an error... need to put global $oldErrorHandler at the beginning of function myErrorHandler(). Also change to $oldErrorHandler = set_error_handler('myErrorHandler');

Page 50:
should read "giving the key for the specific element..."

Page 52:
-- if the two arrays had common elements that also share the same string keys or that have numeric keys (even if they are different) they would only appear once in the end result. This isn't true and should be changed to "if the two arrays had common keys (either string or numeric) they would only appear once in the end result"

Page 53:
var_dump ($a==$c); //should be false

Page 54:
should be echo array_key_exists('a', $a);

Page 54:
should be echo in_array (2, $a);

Page 55:
typo - "note how key key association..."

Pages 56-57: use of reset() is misleading here: only if the $array is passed by-reference to the function, it would be necessary to be reset; otherwise we'll find a brand new reset array inside the function by default.

Page 57:
should be $array = array(1,2,3); ... end ($array); to work with code right after it

Page 67:
"Push and pull" should be "push and pop"

Page 68:
should read "if you intend to use an array as a queue, you can add elements to the beginning and extract them from the end by using the array_unshift() and array_pop() functions".. Also should change the code appropriately

Page 76:
"Here's an escaped backslash - \ - "... should read "...backslash -\\"

Page 77:
echo srtr('abc','a','1'); //(instead of strstr)

Page 79:
strncasecamp instead of strcasencmp

Page 81:
Line 1 should be - "You can use the strspn() function..."

Page 87:
should read "A sign specifier (a plus or minus symbol) ... "

Page 88:
printf("%d", $f); //prints 123

Page 92:
should read "the function returns integer 1 if the match is successful"

Page 98:
should read "Contrary to popular belief, POST is not an inherently more secure way to ..."

Page 99:
2nd par, should read "As you can see, the data has been encoded and appended to the end of "...

Page 99:
should be [direction] not [dir]

Page 99:
Under the [FORM submitted with POST] section, the code says <input type="password" name = "pass' - note the single quote at the end of pass

Page 107:
$data should be $date

Page 109:
Check the "setcookie delete" description

Page 122:
the examples should read public $var1, $var2, $var3 instead of public var1

Page 123:
static public function baz() :: public should come before static

Page 123:
output will display "Hello World... Notice: Undefined property

Page 141:
This chapter says it reviews PDO and database connectivity but it never does... we need to either write something up or take this part out

Page 143:
"one-to-one... correspond to every row in the parent"... should read "each row in the parent"

Page 147:
"suppose we wanted to create a unique index"... but then an ordinary index is created

Page 147:
"isbn ...references book (id)" should say "references book (isbn)"

Page 149:
Ender\'s Game should be escaped with 2 single quotes as such Ender''s Game

Page 153:
sql statement is incomplete, only 2 of 3 tables are joined. Publisher table is missing from the joins

Page 158:
if (is_null(self::$_singleton)) {self::$_singleton = new DB(); - conflicts with the "the getInstance() method, which checks whether the static property $_connection has been iniitialized and, if it hasn't sets it to a new instance of DB

Page 164:
delete this line: function seek($key)

Page 165:
class myData implements Iterator (needed capital I on iterator)

Page 165:
$_myData and $_current needs to remove the underscore

Page 178:
Parsing XML Documents - text is confusing here

Page 184:
$dom->load('library.xml');

Page 184:
should read DomDocument::loadHTMLFile() (capitalize HTML)

Page 185:
DomXpath should read DomXPath

Page 204:
should read " the worst that can happen"

Page 207:
last line, 2nd par - should read "While setting register_globals to Off is the preferred approach"

Page 214:
Reads "Since 1=1 is always true and - begins..." - should read "Since 1=1 is always true and -- begins"

Page 228:
SEEK_CURRENT should be SEEK_CUR

Page 231:
reads "a directory which has the same file as an existing file" - should read "a directory which has the same name as an existing file"

Page 231:
should read "Note that, normally, only the last directory"

Page 231:
should read "Checks if the path is executable

Page 234:
"Stream contexts are created using stream_create_context()" - should read "Stream contexts are created using stream_context_create()"