Feed: PHP, Web and IT stuff - AggScore: 75.3

As a web developer, I am required to build web applications and secure websites. One of the key requirement is to create a secure and well protected system to keep attackers at bay. Although securing the website from malice is important, it is also important to secure the web application from stupidity. Stupidity from the privileged users, administrators, developers, etc. The cost of stupidity is often underestimated. By stupidity I mean lack of average intelligence assumed for a particular task or not thinking through atleast a few worst case scenarios prior to doing something.

Let me give you a few big examples from history and recent times.

1) Data worth 2 billion dollars lost in courier
In 2007, 25 million child benefit records was sent from one government department to another and was lost by the courier company. The discs were sent by a junior staff member unencrypted and unregistered. The discs contained all kinds of personal data, names and ages of children, bank savings account numbers, partners details and even National Insurance Numbers (Equivalent to the Social Security Numbers).  The costs that this caused UK is not clear but it did involve high profile resignations, weeks and weeks of political debates, banks monitoring millions of accounts for fraudulent activity, etc. Some estimate the data alone could be worth over $2 billion in criminal hands.

2) The Honda Point Disaster (Off California Coast)

Aerial view showing all seven destroyers

This was the largest peacetime loss of U.S. Navy ships in which seven destroyers were lost. This tragedy was not caused by malice. Twenty-three sailors died in the mishap. Two other destroyers were slightly damaged. The navy court ruled it as a fault of the navigators and the captains of each ship. How did this happen? The flagship was equipped with a radio navigational receiver, but ignored the bearings, believing them to be erroneous. No effort was made to take soundings or depth measurements. These operations were not performed due to the need to slow the ships to take readings. The ships were performing an exercise that simulated wartime conditions, hence the decision not to slow down. In this case, the dead reckoning (method of estimating position) was wrong and the mistake fatal. The need to slow the ship might also reminds us of another disaster, the Titanic!

3) Aviation Accidents
Boeing studied commercial jet accidents (not including hijacking, test flights, etc) between 1959 to 2008. They determined the primary cause of Airline hull loss accidents (aircraft beyond repair) to be the following:

Clearly a lot of these are preventable and a lot of lives could have been saved. Flight crew error and the Air traffic control accounts to about 60% of all hull loss accidents.

In Conclusion
Accidents do and will happen, I would recommend developers to think about stupidity and not just malice when building systems. I have two interesting quotes to leave you with.
Albert Einstein - Two things are infinite: the universe and human stupidity; and I’m not sure about the universe.
Hanlon’s Razor - Never attribute to malice that which can be adequately explained by stupidity.

Do share incidents that you have come across during your career.

There are many people who have a lot of interesting ideas in mind but never get around to starting on their side/pet project. Some people start on their side project but never get to completion. These are the three steps which if followed before you start actual work on your project will help you get your pet project launched.

1. Think things through
- This is the most important step. If you do this well, consider your side project well begun and half done!
- Visualize things after completion will it make you happy? Will it make you feel like you accomplished something? Will it make you feel successful?
- Humans operate with pain-pleasure principle.  Will the completion of the project alleviate your pain or others pain? Will it bring some pleasure to the work and lives of others or yourself?
- When the going gets tough with your pet project as it often does, these reasons will help you and give you the strength to complete the project.

2. Set a deadline with detailed plans
- Start by identifying all the steps you need to complete and each stage.
- When can you reasonably finish the project?
- When do you want to launch? When do you want to get your product or service out to the public?
- Make detailed and specific plans, not vague ones.
- Also the most important rule, remember nothing is perfect. You will have to refine after launch and constantly learn from it. Take a look at the first version of Google. Your first launch is the beginning get ready to quickly make changes depending on user feedback.

3. Make commitments
- Tell your friends and your family you are going to launch on the deadline.
- If you had prospective customers enquiring about product, let them know that you are launching the product or service on the launch date.
- Schedule banner ads, ppc ads, book presentations, print leaflets, etc as required to promote your project and get you to focus on the launch.
- Get a mailing list of all people who you know that will be interested in the project.
- The third point in my opinion is the one most often ignored and causes deadlines to slip. If  your family & friends know that we are busy working on something they might understand not to bother you until your launch date!

The first point on thinking things through will energize you or show you that this project should never be done. The third point will commit you to the deadline and hopefully clear out distractions as your family and friends are now aware of what you are working on. Only once all the three steps above are complete you should start working on your project and in accordance with your plan.

If you are working on your pet project please do share your experience!

This article uses Amazon EC2 and does not require any additional software installed on your machine to get your own cloud instance running other than your browser. To further configure your instance once it is running you will need Putty or an RDP client depending on your server instance.

Amazon EC2 (Elastic Compute Cloud) is a service by Amazon which lets anyone create, launch, and terminate server instances as needed, paying by the hour for active servers, hence the term “elastic”. The service starts as low as 0.11$ per hour. For this article an instance of LAMP server and a Windows Server 2003 was launched and cost about $0.25. This is very cheap for your on-demand server needs.

Steps to Launch your server instance

1) Signup for Amazon EC2 here (If you already have an amazon account you can save 2 minutes!)
2) Login to the EC2 management console

AWS Management Console (Click to enlarge)

3) On the management console, Click on Launch Instances
4) Select an AMI (Amazon Machine Image). You will get a huge list ranging from windows servers to LAMP servers, Ruby on Rails server, Ubuntu, Debian, Open Solaris etc. If you also look at the community supplied options you will have nearly 3000 AMIs.

List of Amazon Machine Images

5) Create a key pair by following on screen instructions. This is to help you to login to your AMI once it is ready.
7) Configure firewall settings and limit access to the server (SSH, MySQL, Web, etc).

Authorise connection method

8 ) You will now arrive at the final step of your wizard. Enter number of instances. Select an instance type Small, High CPU, etc. You will also have to select the key pair that was generated for you at Step 5. Click Launch once set.

Set Number of Instances

9) Voila! your instance will be available in just a few minutes. Now you can pat yourselves on the back for all the hard work!

It is amazing how quickly you can get an instance running. Once your server is up and running how do you connect to this instance? Well, it is pretty simple if you are using windows you get to use RDP (Terminal Services). For non-windows instances you will have to use SSH to connect to the server.

Advanced configuration of your Windows instance using RDP (optional)

It is easy to connect to a windows amazon instance via RDP compared to SSH connection to a linux server. To retrieve the RDP password, you will have to right click your instance and retrieve connection information using the key pair you have been given.

Connecting via RDP

You will have to connect using Administrator as your username. You will find all other info you need on the management console.

Advanced configuration of your Linux instance using SSH (optional)

You will need putty to connect to your linux server via SSH on a windows machine. If you are on a linux machine you will be able to use the ssh command as shown on the management console. With putty you will also need an additional tool to create a PPK file from the PEM file that you are given from the management console. To do this you will have to download an additional tool called PuttyGen.exe. You have to load the PPK file under the SSH Auth section of the putty client.

SSH Connection to your instance

Once your instance is up and running you will be able to connect to the webserver, connect to the MySQL server, if it is enabled with the AMI. The last two sections are to help you further configure your instance further. There are so many AMIs available with a lot of features already built in. Technically you will have your services running after the first set of steps which can be complete in 15 minutes!

If you do not want to configure a server yourself take a look at the Windows Azure, Force.com or the Google App Engine which runs Java, Python and PHP using Quercus for free (certain usage limits apply).

Since its birth in 1994, Javascript has come a long way. Today it is one of the most popular programming languages on the web. Javascript has been on the rise and is growing faster since AJAX based applications reignited professional developer interest. It is now possible to it to write applications on the server, the mobile devices, on the browsers (add ons and plugins),  inside PDF documents, even in some remote controls and many more to come.

Javascript application development in Mobile devices

Build Native Applications using Javascript

Palm’s new OS is the first mobile platform to be built from the ground up to combine standard technology, innovation and integration. At its core, webOS leverages several industry-standard technologies, including web technologies such as CSS, XHTML and JavaScript. You can think of webOS applications as native applications, but built from the same standard HTML, CSS and JavaScript that you’d use to develop web applications. Palm has extended the standard web development environment through a JavaScript framework that gives standardized UI widgets, and access to selected device hardware and services.

Javascript can be used on the iphone too.  The HTML 5 specification provides a new mechanism for client-side data storage: JavaScript database support. This feature shipped originally with iPhone OS 2.1. When you use “Add to Home Screen” from the “+” button on Safari for iPhone, a web application with a manifest defined (per the HTML5 spec) will be saved with any cached resources.  It is also worth noting that these features have not been added to Safari on Mac OS X or Windows.

Desktop Application development using Javascript

Adobe AIR is a cross-platform desktop runtime created by Adobe that allows web developers to use web technologies to build and deploy Rich Internet Applications (RIAs) and web applications to the desktop. This means what you can write you application using Javascript and then let AIR to figure out how to run your code in Windows, Linux, Mac and future operating systems.

Desktop App using Javascript

The application shown in the pic is written using Javascript, HTML and CSS. It uses the ExtJS library. Javascript can be used to dock the application in the taskbar, delete and modify files on the hard drive, communicate to different server, use AJAX, etc. You can even use flash within your app to make the application look and feel much better. This gives the web developer good access to a desktop.

Server side Scripts using Javascript

Server side JavaScript has been around for a long time and potentially offers some unique and interesting advantages over other languages (like PHP, ASP, etc) because the same language is spoken by both client and server. There are numerous attempts to bring server side javascript to the masses.  Server side Javascript has to do much more than what it has to do at the browser level with access to databases, files, and networking, as well as logging, process management, scalability, security, integration APIs, and extensibility. You can even use your own javascript libraries on the server jQuery, dojo, Ext JS, prototype, etc.

Javascript in other places

- Adobe reader (from v3.02) supports Javascript for forms and basic operations. SOAP support has also been included since version 7.0. There is also access to the entire 3D Javascript API. More details on using Javascript is available here
- Open office application suite supports Javascript to write macros. This move looks like a sensible addon given the usage of VBScript in MS Office suite.
- Apple’s Dashboard Widgets, Microsoft’s Gadgets, Yahoo! Widgets, Google Desktop Gadgets are implemented using JavaScript.
- The oddest one in this list will have to be the  Philips Remote Control which uses Javascript

ECMAScript 4 (ECMAScript Harmony)

Javascript has been relatively stable since the third edition of ECMAScript published in 1999. Features under discussion for a future edition originally ECMAScript 4 now ECMAScript Harmony include Classes, a module system, static typing and Optional type annotations, Generators, Iterators, Destructuring assignment, algebraic data types. Well if you think this is ambitious you should see the original ECMAScript 4 which had more semantic and syntactic innovation. Packages, namespaces and early binding from ECMAScript 4 are no longer included for planned releases with ECMAScript Harmony. The intent of these features is partly to better support “programming in the large”, and to let programmers sacrifice some of the script’s ability to be dynamic for performance.

So yes as a web developer who uses Javascript on a daily basis, I am excited to watch Javascript grow and be available in more places.

A gotcha in programming is something unexpected although a documented feature of a computer system and not a bug. These gotchas throw beginners off Javascript programming and send some people running and screaming. In my opinion, Javascript is one of the most widespread languages as it is available on all browsers but one of the least learned languages.  Let’s start with basic ones.

1. Floating point arithmetic

This can be a major cause of frustration for people who are new to Javascript and are trying to perform some kind of mathematical computation.

alert(0.02 / 0.1);  //0.19999999999999998 
alert(1.14 * 100);  //113.99999999999999    ;)

This is where the Math.round() functions come in handy

2. Plus operator overloading 

The plus operator stands for both arithmetic operations and also string concatenation. This is convenient if used correctly. Lets take a look

var msg, one="1";
msg = 2 + "1"; // msg = "21"
msg = 2 + one; // msg = "21"
msg = 1 + 1 + 1 + " musketeers"; // msg = "3 musketeers"
msg = "Bond " + 0 + 0 + 7; //msg = "Bond 007"  

 
The above behaviour is because the operations are performed left to right. If we use parantheses the behaviour will change based on the order of the string or number in it.

3. Semicolon insertion at line feed

Javascript automatically inserts ; at a line feed. Lets see this in action with a simple example:

function returnSame(a){
   return                 //Inserts semi-colon to convert to return;
   a                      //a becomes a; - Unreachable
}
alert(returnSame(2));  //Output is undefined

The magical semicolon insertion can get things complicated while creating objects using object literals

4. typeof operator

typeof is a unary operator. The results are not one would normally expect. It suprisingly evaluates null to ”object”. 

var obj={};  //object created using object literal
var arr=[];  //array created by array literal
alert(typeof(obj));   //object  - Good
alert(typeof(arr));   //object  - Bad
alert(typeof(null));  //object  - Ugly!  ;)

 
It should only be used to distingush objects from other primitive types.

5. false, null, undefined, NaN, Infinity

They stand for different things although they might look similar or at times look redundant. Javascript uses three primitive datatypes numbers, strings and boolean. In addition it has two trivial datatypes undefined and null. Both null and undefined are equal according to the equality operator (==)

 

var a;
alert (a);    //undefined
alert (1/0);  //Infinity
alert (0/0);  //NaN
0/0 == 0/0;   //false - a NaN != NaN
alert (b);    //error

6. String replace only replaces first occurence

Unlike PHP or other languages, string replace in Javascript only replaces the first occurence of a string by default.

var nospace = "I dont need spaces".replace(" ","_");
alert(nospace);    //I_dont need spaces   - Only first occurence
var nospace = "I dont need spaces".replace(/ /g,"_");
alert(nospace);    //I_dont_need_spaces

7. ParseInt function

This is used to convert a string into an integer. The function does accept two arguments and the second argument specifies the radix. The radix here for decimal is10. If no radix argument is passed the parseInt function tries to guess the base and in this case because the string starts with 0, it is parsed as an octal number.

var str = "017";
var strInt = parseInt(str);      //strInt = 15  ;)
var strInt = parseInt(str,10);   //strInt = 17

Please share your Javascript Gotcha’s using the form below:
(note use < and > to get past the comment filter for )

Both FTP and SMTP are simple text based protocols. A previous article showed how to check if an email address exists using SMTP commands from the terminal. Here I would like to show you how you can use raw FTP commands to connect to an FTP server, login, traverse directories and even download files. But before we do this we need to understand how FTP is different from the other protocols. 

Firstly FTP (File Transfer Protocol) uses two channels, the data channel and the control channel.  This is called out-of-band control. The control channel sends commands to the FTP server and the data channel is used for data (to retrieve files from the server, etc).

Secondly there are two major modes of FTP operation, the active mode and the passive modes. The difference lies in the way the data channels are opened. In Active FTP, the FTP server will connect to the client port and send data to it. In Passive FTP, the FTP server will tell the client which port to connect to for retrieving data. Firewalls can complicate the process on both sides. 

In our example, we will use Passive FTP (avoiding firewall issues on client) to download a file using anonymous FTP login to the IETF servers. There are a lot of files on this server by some estimates it is about 4GB. We will pick up a small file called ftpext-charter.txt located in the /ietf/ftpext/ folder on the server.

Open the terminal/command prompt (On windows, Go to Start > Run > type cmd). Once you are on the command prompt, type this command to connect to the FTP server and issue commands

C:\> telnet ftp.ietf.org 21

220 ProFTPD 1.3.1 Server (ProFTPD) [64.170.98.33]
USER anonymous
331 Anonymous login ok, send complete email address as your password
PASS blogger@webdigi.co.uk
230 Anonymous access granted, restrictions apply
CWD ietf/ftpext/
250 CWD command successful
PASV
227 Entering Passive Mode (64,170,98,33,151,31).
RETR ftpext-charter.txt
150 Opening ASCII mode data connection for ftpext-charter.txt (6060 bytes)
226 Transfer complete
QUIT
221 Goodbye. 

                                     Commands/Response on control channel

We issued these five commands in the following order at lines 2, 4, 6, 8, 10 and 13. 
USER - Send username to the FTP server
PASS  - Send the password (Anonymous servers need email address)
CWD  - Change the working directory on the server
PASV - To enter the passive mode (To let client connect to the server)
RETR - To retrieve a remote file from the server
QUIT - To terminate the connection to the server 

Between line 10 and 12, you will notice that the file was downloaded. To start the download, I had to open up another telnet window to open the data channel. To figure out to which IP address and port I had to connect to, we have to look at line number 9. We received a set of numbers (64,170,98,33,151,31) from the server in response to the PASV command. The first four related to the IP address 64.170.98.33 and the last two 151 and 31 help us identify which port to connect to.  Multiply the first by 256 and add it to the second. So, 151 * 256 + 31 which is equal to 38687.  Now that we have the IP address and port number, all we have to do is to open a second terminal and telnet to IP:Port as shown below:

C:\> telnet 64.170.98.33 38687 

This will now show you all the contents of the file ftpext-charter.txt being thrown into your second terminal window. Once this is done, you can proceed to type further commands on the control channel (the first terminal window).

Notes:
- The anonymous FTP server on IETF has a 60 second timeout on its control channel connection. Please connect to your own FTP servers they might be more forgiving to humans on terminals.
- FTP is not very secure as you can see the password and username are sent in plain text! Also, there is no encryption as you saw on file downloads or uploads.
- Type HELP once you send your password to see what commands you can issue the server.
- Here is a list of raw FTP commands and the parameters
- Here is a list of anonymous FTP servers
- This is the FTP sequence diagram which explains stuff at DNS and TCP level
- On windows there is a built in command line FTP tool (called ftp). It is useful but it does not show us how to use raw commands and communicate to an FTP server.
- SFTP (SSH File Transfer Protocol), FTPS (FTP over SSL) are more secure ways of using FTP.

The FTP HELP command via terminal

Hope this helps!

Google launched their Google App Engine (GAE) a year ago. The free hosting in App Engine is allocated 500 MB of persistent storage and enough CPU and bandwidth for about 5 million page views a month. Also, if you really want more you can see pricing plans.

GAE will support Java going forward. Unfortunately PHP support on the App Engine is still left as the top item in the wishlist. So until Google announces their official PHP support we have a workaround to run PHP using Quercus. Quercus is basically a 100% Java implementation of the PHP language (requires JDK 1.5).  Since the App Engine now supports Java this means we can use Quercus to run PHP scripts on the App Engine.

So all you need to use the GAE and run PHP
1) Register a free account.
2) Download this file to your computer.
3) Edit application XML tag in the file war\WEB-INF\appengine-web.xml to the name of the application you have registered.
4) Finally upload your application. I downloaded Google App Engine SDK for Java and use the following command in windows.
appcfg.cmd update C:\projects\phpwithjava\war

To see this in action just visit:
http://phpwithjava.appspot.com/webdigi.php and http://phpwithjava.appspot.com/info.php

NOTE: phpwithjava is my app name with GAE.

Around 80% of the end-user response time is spent on the front-end. A fair share of this time is spent on downloading components of the page like scripts, Flash, stylesheets, images etc.  Javascript takes majority of the loading time of a webpage because scripts block parallel downloading and rendering in the page. Even if you do not have a lot of Javascript files to load on your webpage they can still block loading other content on your page while they load.  Lets have a look at how the standard javascript file include method and the script DOM element method in detail below.

Standard Javascript file include method

Javascript blocks the other elements from loading (Example)

Script DOM element method

var p = g.getElementsByTagName("HEAD")[0];
var c = g.createElement("script");
c.type= "text/javascript";
c.onreadystatechange = n;
c.onerror = c.onload = k;
c.src = e;
p.appendChild(C);

Javascript executed without blocking any element from loading

This method creates a DOM element for each Script and then adds the element to the HTML. 

NOTE: 

1) Only when the appendChild function is called the Javascript will be executed.
2) For inlined code that depends on the Javascript and also for a similar method of Asynchronous Script Loading - See Steve Souders blog
3) If you go to MSN.com (Alexa top 10 site based on traffic) and hit view source you can see the javacript elements are included by using the script dom element to load the web pages faster. This is a good example of where you can use the script dom element method for certain js files that do not have inlined code dependency. Let us run a Pagetest waterfall report and you can see the following for MSN.COM

Pagetest Waterfall report of MSN.COM shows no blocking during page load

In a recent PHP conference in London some great speakers spoke about new features in PHP to be released in PHP 5.3. PHP 5.3 contains functionality that was scheduled for PHP 6, which takes PHP 5.3 from being a minor release to a significant and huge release. A release that no PHP developer should ignore. Most of these features are pretty complicated additions for novice PHP programmers. I have listed some features and some ways to use them.

1) Namespaces for classes and functions
This feature will help us shorten the class names and function names. To appreciate this feature, we need to go back to the days before there was Object Oriented Programming in PHP. Imagine all the function names with name save(). How would you differentiate if the call save() was to save a blogs or save comments? The solution was to use blog_save() or comment_save() before the introduction of classes in which we could write the save() function within the Blog class or the Comment class. Using classes is obviously a much more elegant solution.

We now have the same situation with the large number of classes and functions. Using namespaces, we could simply separate the two functions above in the code below:

EDIT: A final decision was made on October 2008.  Developers will have to use \ backslash operator to dereference namespaces.

2) MySQL Native Driver
PHP 5.3 has a native driver specific to PHP, optimised for the ZEND engine. It is an alternative to connect to MySQL server versions newer than 4.1. Being a native driver we should be able to get much faster execution times. The native driver will also be licensed under the PHP license. If you are like most users, you are currently using libmysql (A MySQL database client library) you will be able to easily switch over to mysqlnd without making any changes to your existing PHP Scripts!

3) phar - PHp ARchive
This is a cool new feature.  Think of it like an archive, like a .zip file or a .tar file. Besides just being able to group all the files into one simple file, we will be able to deliver and run an entire PHP application from a single file!

We will also be able to use phar archives within PHP, so the following will work in PHP 5.3 and above

Obviously, there will be a performance hit but the possibilities are endless, imagine being able to upload phpMyAdmin to the server as a single phar file instead of hundreds of small files.

4) Closures & Lambdas
This gets into the list because this is something most web developers would have been familiar with while working on Javascript. A lambda can be declared anywhere and they can be assigned to a variable. A closure on the other hand are lambda funcions but have access to the variables where they were declared. This is something called lexical scoping. To see this in action take a look at this example.

5) All of the rest!
There are a lot of other things in PHP 5.3 which I thought are nice, I have just described all of them very succinctly.
Functors: This allows an object to be invoked as a function.
Traits: This is a new unit of reuse, traits can be incomplete, provides reusability, modularity and structure. In short it is copy-paste glorified!
Magic functions: We have a couple of new magic functions for classes (interceptors) __callstatic() and invoke()
Ternary operator: You can now display the a value that exists $value1 or $value2 using this simple statement echo $value1?:$value2;
There are many more things added like Late Static Binding, Variable Static Calls, Changes to PHP Error Levels, new PHP functions, improvements to help with OpenID, Command line and many more.

Final Thought
Well, this gives us much more to play with. It is definitely a lot to include into PHP 5.3 and I would have expected so many changes to go into PHP 6. I sometimes wonder if there will be anything new left to add into PHP 6 given the fact that so much has been released already.  If you are interested in PHP 5.3, do give it a try here, it is in beta at the time of the writing.