Originally Published: Monday, 26 March 2001 Author: Marcelo Pham
Published to: featured_articles/Featured Articles Page: 1/1 - [Printable]

Writing Database Oriented Web-based Applications with Perl - Part IV

Marcelo wraps up his series with some tips on debugging web applications, security, and advice on programming standards.

   Page 1 of 1  

Interfacing Perl with the database (II):

19. Editing and deleting records from your application
20. Debugging your web-based application
21. A little bit about security
22. Introduction to Programming Standards
23. Books
24. A thought about this series of articles

19. Editing and deleting records from your application:

In the last installment we made a simple script to add a new record to the contact table. We will now make a simple program to modify and delete an existing record. Please recall the sample scripts of the last installment, that is, add.html, add.cgi and add2.cgi.

Let's make the edit screen. As usual, the HTML page (edit.html ) goes first:

<html>
<title>Edit Contact</title>
<br>
<form method='post' action='cgi-bin/edit.cgi'>
Edit Contact:<br><br>
Enter Contact Code: <input type='text' name='code'><br>
<br>
<input type='submit' value=' Edit '>
<input type='reset' value='Reset Form'>
</form>
</html>

And the cgi scripts go after. First, edit.cgi:

#!/usr/bin/perl
# Edit.cgi: script that edits a contact (from edit.html)
# Required libraries

use DBI;
require "parseform.lib";
require "dbconn.lib";
# Parse fields from form
&Parse_Form;
# Pass into local variables:
my $code=$formdata{'code'};
# Mime tag
print "Content-type: text/html\n\n";
# Validates code:
if ($code eq '')
{ print "Contact code cannot be empty!\n"; exit(0); }
# Inquiry current values from database
# Connect to the database
my $dbh=DBI->connect(&connectionstring;&dbuser;&dbpassword);
my $sql="select * from contact where code='$code'";
$sth=$dbh->prepare($sql);

$sth->execute;
# Disconnect from database
$dbh->disconnect;
# Pass result to local array
my @rows=$sth->fetchrow_array;
# Verify if entered code exists
if ($rows[0] eq '')
{ print "Entered code does not exist!\n"; exit(0); }
# Print screen with current values:
print "<title>Edit Contact:</title>\n";
print "<br>\n";
print "<form method='post' action='cgi-bin/edit2.cgi'>\n";
print "Edit Contact:<br><br>\n";
print "Contact Code: <b>$code</b><br>\n"; # Key. Cannot be changed
print "<input type='hidden' name='code' value='$code'>\n"; # Hidden code
print "First Name: <input type='text' name='first' value='$rows[2]'><br>\n";
print "Last name: <input type='text' name='last' value='$rows[1]'><br>\n";
print "Phone: <input type='text' name='phone' value='$rows[3]'><br>\n";
print "<br>\n";
print "<input type='submit' value=' Update '>\n";
print "<input type='reset' value='Reset Form'>\n";
print "</form>\n";
# End of script

And now, the last script (the one that actually updates the database) edit2.cgi:

#!/usr/bin/perl
# Edit2.cgi: script that updates Contact info (from edit.cgi)
# Required libraries

use DBI;
require "parseform.lib";
require "dbconn.lib";
# Parse fields from form
&Parse_Form;
# Pass into local variables:
my $code=$formdata{'code'};
my $first=$formdata{'first'};
my $last=$formdata{'last'};
my $phone=$formdata{'phone'};
# Mime tag
print "Content-type: text/html\n\n";
# (Validation code goes here)
# Update database
# Connect to the database
my $dbh=DBI->connect(&connectionstring;&dbuser;&dbpassword);
my $sql="update contact set first_name='$first',last_name='$last',

phone='$phone' where code='$code'";
$sth=$dbh->do($sql);

# Disconnect from database
$dbh->disconnect;
# Inform user
print "Contact info updated successfully!\n";
print "<a href='edit.html'>Edit another contact</a>\n";
# End of script

Briefly:

  • The HTML page (edit.html) prompts for a contact code and calls edit.cgi.
  • edit.cgi retrieves information from the database for the entered contact code (and stores it in a local hash called @rows ), and generates an HTML page displaying current values for all the fields (please notice how I pass the primary key code -which cannot be modified- to the third cgi script using a hidden field).
  • User can make changes in the screen, and the form executes edit2.cgi when the user submits the page.
  • edit2.cgi grabs all the new information, parses the form, retrieves the new/modified values and construct the SQL statement to update the table.
  • Finally edit2.cgi updates the database using the method do (since you don't need any query result), constructs the SQL statement and updates the database with the new values (if no new values were entered, the script will update the database anyways).

I will leave the delete script for you as an exercise. (solution available)

This is basically what you have to do for each function of your web-based application, you can use these skeletons to build your own. Remember that you are in a high traffic environment and every single byte that you send or receive affects the whole application's performance. And also remember to implement your code based on your initial design.

20. Debugging your web-based applicati on

Assuming that you understood the structure of the programs, let's go over a couple of very basic debugging techniques when programming web-based applications.
Remember, these are just guidelines but the truth is that debugging depends mostly on the programmer's common sense and imagination!

Syntax errors: your first attempt to find possible typos in your script is compiling the programs for syntax errors. You can achieve this by using the Perl compile option (-c). From your Linux console type: # perl -c <myscript.cgi>

And Perl will output errors and/or warnings. If everything is fine, Perl will output <myscript.cgi> syntax OK

File errors: usually you cannot execute a Perl file if it does not have permission to be read and executed (for example, when you get a "Internal Server Error" page from your browser). Make sure that your script can be read and executed (using chmod a+r <myscript.cgi> and chmod a+x <myscript.cgi>) and, if you are working from a workstation, make sure that the script was properly uploaded (in ASCII mode).

Library/Module errors: you may also get an annoying "Internal Server Error" page if you have a script that declares a library or module that does not exist. For instance:

#!/usr/bin/perl
use DBI;
require "dbconn.lib";

In this case, either DBI is not installed or dbconn.lib does not exist or cannot be located under the script directory.

Database errors: to trap database operation errors, DBI returns two variables: DBI::err and DBI::errstr in order to describe the error. For example

$sth=$dbh->do($sql) or die "Error $DBI::err = $DBI::errstr";

What you can also do is include the SQL statement to see if it was correctly constructed (try the SQL statement from your database shell utility to see if it also gives you an error).
A last resource would be the Apache log file (please see below).

Variable/Function name typos: you should look for consistency of function and/or variable names. For instance, if you have

my $div=$formdata{'div'};
# more script lines
my $finalresult=$age/$dib;


This will give you a "Divide by zero" error, because you typed $dib instead of $div. Use the search function of your favorite editor to verify that your variable and function names are the same throughout all your script.

Commenting lines to isolate the error: if you are getting errors and cannot find out the reason, you should start commenting lines to isolate the error:

my $div=$formdata{'div'};
# more script lines
# my $finalresult=$age/$dib; # I am commenting this line !

Start commenting the lines that are suspicious. Then, keep commenting until the script does notgive the error. When you reach that point, the last line or lines that you commented may be the source of the error.

Watching a variable's progress: to watch a variable's behavior, you can add lines to print the different values that are assigned to your variables during the execution of your script. For example,

my $div=$formdata{'div'};
print "1 - Div = $div\n";
my $div=$div*100+0.32334;
print "2 - Div = $div\n";
my $finalresult=$age/$div;
print "3 - Div = $div -- FinalResult = $finalresult\n";

Do not forget to print a reference to know where in the program the variable is getting the printed value, and to add the Mime tag before start outputting to the browser!

Debugging with the Apache file log: finally, you can use the output log of the httpd daemon to know what went wrong. From your Linux console, type:

# cd /var/log/httpd
# more error_log


or

# more access_log

Apache stores a log of what was wrong when trying to output to the remote browser, and normally the error descriptions are quite informative, so you will not have any problem figuring them out.

These are just a few basic techniques that you can use separately, but you must combine them at your convenience to obtain the best and quickest result.

21. A little bit about security

These are my two cents thoughts about security issues; you should try to have these in mind when building your application:

  • Try to come up with a function that controls security, ideally a function that can be called from a library, and call this function from the very beginning of each script. If you have to change the security algorithm, you will always go to only one script, not all of them.
  • If you use cookies to control and monitor sessions, make sure that the cookie expires when the session does (that is, when the remote user's browser is closed). Never write cookies with the same content, always try to store different values for different sessions (like timestamps, remote IP address, etc.). Perl has environment variables -stored in a hash- that allow you to retrieve some information about the remote computer (for example $ENV{'REMOTE_ADDR'} will return the remote IP address which is usually dynamic). Please refer to your Perl documentation for further information.
  • Try to come up with a security algorithm that can be easily and quickly changed. If you detect that someone is attacking or trying to obtain a password with a brute force script, you should be able to rapidly change the logic and structure of your security policy.
  • You can also protect the directories where the CGI scripts are stored, with the htpasswd utility. Please refer to the man pages for further information.
  • To avoid someone intercepting your passwords, you can also use the Secure Socket Layer (SSL) in the login process. Please refer to your Linux documentation or browse our how-to guides in order to implement it.

22. Introduction to Programming Standards:

Back in the seventies, when computer applications were starting to come out, computer programmers began to code in the first languages such as Assembler, COBOL (COmmon Business Oriented Language) and Fortran (FORmula TRANslator). Obviously the computers that ran these programs were old, big, not so powerful machines, and only affordable by big companies such as Bull and IBM. So, they basically used business applications and therefore had to hire teams of programmers in order to maintain and enhance their software. Programmers then had to come up with a collaboration environment so they could understand what they did and standardize the code and procedures in order to optimize the programs and maximize the team efforts.

The computers were not as powerful as today's, memory and storage media were expensive and compilers and interpreters had very limited stack and heap sizes, so programming a complex application efficiently was a hard duty to carry out. Standards were necessary. They learned the hard way, but left us a very good legacy.

Standards will make the application documentation, debugging and enhancing much easier. Also, if you decide to sell it or if it becomes an open source project that will have thousands of programmers involved your use of standards will be essential.

We will go over the basics in standards programming. This is just a summary from my point of view and my experience programming in different platforms and languages. It may differ from other coders' opinions, but I think these are the most important points to have in mind when developing any software application.

System design: as I stated in the first installment, system design is absolutely necessary before you start programming. Whatever design standard you choose, stick to it until the end of the development.

Naming convention: try to come up with a naming convention for programs, libraries, variables, functions, procedures and database elements.
The name should be closely related to the functionality of the object (program, variable, library, table, etc.). Always remember that you will not remember every little detail of your application code two years from now, so if you name an update program "u.cgi" you will remember it for the next 10 hours, but you will not in a couple of months (it should be, for example "update.cgi").
The naming convention could be any as long as makes sense and you keep it constant throughout all the implementations. For instance, programs can have the extension ".pl", libraries ".lib", etc., and the name must start with the module name and the table name followed by the main function of the script, all separated by underscores (like "ar_customer_update.cgi", for Accounts Receivable Customer update program).

Program structure: structure your program as if it were a cheesecake recipe. Differentiate the different sections of your program (libraries, variables declaration, connection/disconnection to and from database, business logic). The Perl interpreter is flexible and allows you to organize your program the way you want it, but for example COBOL has well defined sections for different tasks. (I can remember some! Identification division : author, program name declaration; File section: file opening; Working-storage section: variables and functions declaration.) You should do the same -perhaps not exactly the same way- with your Perl script. Messy programs are hard to understand, debug and document.

Program comments: you should include two types of documentation in each source script: the header and the detail lines. The header should contain the program name, author, company name, brief description of its functionality, notes, dependencies, and dates for creation and future modifications.
Then, the detail lines should be located in strategic points of your code, briefly describing what the portion of the program does. When I say briefly, it may be interpreted in different ways. Do not describe too brief, like:

# update

and do not be too extensive either, like:

# I am updating here the table contact with the values that I retrieved from the prior form, which is in the file called edit.html

You are really the one who 'measures' this, but for me a good comment line would be:

# update contact table (from edit.html)

Functions and procedures: when you design your application, you will notice that a lot of functions can be resumed and concentrated in one procedure that can be called from different programs. Or, the good programmer will try to abstract as many functions as possible and convert them in one callable with different parameters. This function or callable is basically a function or procedure that will behave based on the parameters that are passed from the program. A function usually accepts parameters and returns values; a procedure accepts parameters but only performs tasks.

What is the advantage of this? First, reusability . For future programs, you will not have to reenter the code -or cut and paste-, you simply call the function or procedure with the correspondent parameters. Second, easier debugging. If you get an error, you just debug the function or procedure, not all the programs that call them, therefore, if you fix it, you will be fixing just once. Third, enhancements . If you want to enhance or modify the behavior of a function or procedure, you just do it once and it will reflect the changes in all the programs that invoke it. Fourth, disk and memory usage. I can keep going but I believe you get the picture...

Libraries: once you have all your functions defined, you will probably store them all in a library to separate them from the main scripts, for easier reading and better deployment. Now, you will also notice that some functions or procedures are applicable only to certain modules. A smart thing to do is to separate or 'categorize' functions and procedures in distinct library files. For example, you could create a library called 'inventory.lib ' containing functions and procedures that are called by all the Inventory module programs.

First, for organization and debugging purposes . Second, to not overload the Perl interpreter (from your script, just declare the libraries that contain the function and procedures that you will use, not more, not less) and improve the application performance.

Some other considerations: whatever you do, do it consistently. If you decide to use the ".pl" extension, do it for all your scripts. If you decide to use long name variables, apply them in all your scripts.
I am a 29 years old programmer and have learned from the COBOL and Assembler generation of programmers. Old generation coders may agree with my opinion, new generations may differ, but the important thing is that there should always be standards when developing an application.

23. Books:

Recommended reading:

  • "The Mythical Man-Month: Essays on Software Engineering" (Frederick P. Brooks Jr) - Addison Wesley
  • "Software Engineering" (Ian Sommerville) - Addison Wesley

24. A thought about this series of articles :

Throughout all these articles I've shown you how to design and implement your web-based application. If you look at it in a generic way, it is really applicable to any other kind of application, and for any kind of operating system and programming language.

It does not matter if your favorite platform is Unix, Linux, Windows, OS/2 or Mac, and it does not matter if you like Perl, C, Java, Visual Basic, or COBOL either. What matters is your passion to develop software that will help you and others to achieve what technology revolution promised twenty years ago: to make our lives easier.

I consider programming as an art. The programmer is an artist who expresses himself through programs that come from his abstraction of a given problem. The good programmer thinks that there will always be a thousand different ways to solve the same problem. "You're not born as a programmer, you make yourself a programmer" they say. This means, skills come with experience. Try to do as many developments as possible, just for fun or for your job; enjoy the challenges, and always think that there is a better way to do it, no matter if you thought about it for months -be a hardcore non-conformist!. Experience will give you expertise, speed and a more exercised brain.
Also, keep yourself updated with the latest technologies, and don't be close-minded with an operating system, programming language or database engine. It's okay to have a favorite, but always keep your mind open to new technologies and be prepared to change if you have to...

Some problems can be solved with Linux and Perl, some others might be solved with Windows and ASP scripts, or it may be a situation where you have to use the best of both worlds, or look for a different one! Common sense is the programmer's best buddy, and, as my ex-professor of networking used to say, "do not kill an ant with a bazooka, measure your resources" (thanks, Omar!).

I hope you enjoyed this series, and I also hope that it helped you in some way. As always, if you have a question or want to contact me, please feel free to do it by email.


Summary:

  • Use the skeletons described above to build your own application, always based on your initial design
  • There are many ways to debug your application. Don't use them individually, try to combine them to obtain best results
  • Programming standards are a must in a programmer's bible

Marcelo Pham is a writer for the Linux.com Develop Section. Marcelo is a systems consultant who was born in Buenos Aires, Argentina, but lives now in sunny Miami, FL.





   Page 1 of 1