Pretty Good Combobox HTML Implementation

I saw this a couple months ago and love it. It is a drop down that you can enter text directly in. Pretty clever, they used CSS to put a textbox on top of a drop down. Then using a little JavaScript, the two become one.

Here's the link: http://particletree.com/features/upgrade-your-select-element-to-a-combo-box/

News About Programmer Jobs

I read in the October 16, 2006th copy of Information Week, that jobs for coders is starting to stabilize. The article has a dark view on this news, mainly because the amount of jobs is not going up. However, I see it differently. Overall, since 2000, the number of employed programmers has dropped about 200,000. While the number of people who considered themselves programmers, has dropped as well. I started my professional career in 1998 as a Java developer for an IT consulting firm. We got torn to shreds when the dot com crash of 2000/2001 hit. I was one of the lucky ones who managed to stay employed. I saw a lot of friends out of work. Every year it seemed like things were getting worse.

So, now, with the programming jobs becoming more stable, things look much better. I would rather that the jobs remain stable, then go down hill. The article lists updating legacy systems, customizing apps, and writing new apps as positions where developers are in high demand. One thing is certain though, technological progress will continue, therefore, programmers, and software engineers, will always be needed.

Reading Files With ColdFusion, or Java???

One project I was on, not to long ago, consisted of transferring a datafile between 2 systems. Unfortunately, this datafile wasn't XML. It was a pipped delimited file of product records. The main purpose of this file was to sync up data between the two systems. Therefore, I needed to read the file, line by line, parse out each field of the file, and compare it with current records in the database. Since the language of choice for this particular project was ColdFusion, I was encouraged to keep with the standard.

Being a Java guy, with some php and perl background, I knew I could adapt. But how can I make this as efficient as possible. The app server we were using was ColdFusion version 6.1. So, I know I had options. At least it wasn't version 5 or below, like some other organizations who shall remain nameless.

Anyway, ColdFusion has a tag called <cffile>. This tag is nice and easy to use. However, it has one main drawback to it. It holds the entire file in memory at once. It might not seem like a problem, but I am talking about somewhere around 200,000 records. With each record containing 80 fields. That can make the application more prone to memory errors. Especially, if this is a user requested job, and not a scheduled job.

However, the standard Java development kit has the FileReader, used with a BufferedReader, that can stream the file, line by line. So, I figured I'd give it a try. I started by comparing the two methods of file reading. I wrote a sample ColdFusion page which would read the file in and compare the time it takes to read/process the file using the <CFFile>, and the Java FileReader methods. I started with a small file, containing 200 records. Then moved up to a file with 25,000 records. The code is as follows:

Using cffile:
<cfset filePath = "./DataFile200Records.txt">
<cffile action="read" file="#filePath#" variable="fileContents">

Using CFFile<br>
<cfoutput>
Start:#now()#<br>
<cfloop index="line" list="#fileContents#" delimiters="#Chr(10)#">
  <!--- Account for empty values --->
  <cfset newline = " " & Replace(line, "||", " | | ", "all") & " ">
  <!--- Convert the pipped delimited list to an Array --->
  <cfset fieldArray = listtoarray(newline,"|")>
  <cfloop index="i" from="1" to="#arraylen(fieldArray)#">
     <!--- Loop through each field --->
     <!-- #fieldArray[i]# *** -->
  </cfloop>
</cfloop>
End:#now()#<br> <!--- End Timestamp --->
</cfoutput>

Using Java FileReader:
Start:<cfoutput >#now()#</cfoutput><br><!--- Start Timestamp --->
<cfscript>
// create a FileReader and BufferedReader objects
fileReader = createObject("java","java.io.FileReader");
buffReader = createObject("java","java.io.BufferedReader");

// Instantiate the FileReader with the file path
fileReader.init(filePath);

// Instantiate the BufferedReader with the fileReader Object
buffReader.init(fileReader);

// read the first line into a String
fileLine = buffReader.readLine();

// Loop while the String is defined
while (isDefined("fileLine")) {
// Account for empty values
newline = " " & Replace(fileLine, "||", " | | ", "all") &amp;amp;amp;amp; " ";
// Convert the pipped delimited list to an Array
fieldArray = listtoarray(newline,"|");
for (i = 1; i lt arraylen(fieldArray); i = i+1)
{
  // Loop through each field
  writeOutput("<!--" & fieldArray[i] & " *** -->");
}
// read the next line to continue the loop
fileLine = buffReader.readLine();
}

// close the Reader objects
buffReader.close();
fileReader.close();
</cfscript>
End:<cfoutput >#now()#</cfoutput><!--- End Timestamp --->

Essentially, each section of code is doing the same thing. It reads the file, line by line. Parses the fields, including empty fields. And it loops through each field, displaying it as an HTML comment. The main difference is that the Java FileReader and BufferedReader objects have to be closed when they are no longer needed. You always want to close a Streamed object, otherwise it can take up memory.

So, what are the results? You would think that CFFile would run faster, being that the file is in memory. And memory access is usually pretty quick. Well, for the small, 200 record, file, that is correct. Below are the results:

200 Records

Using CFFile:
Start:{ts '2006-11-03 10:23:32'}
End:{ts '2006-11-03 10:23:33'}

Using Java FileReader
Start:{ts '2006-11-03 10:23:33'}
End:{ts '2006-11-03 10:23:35'}

Using the Java objects took a second longer. I think that may have to do with the overhead involved with opening and closing the Reader objects. But that would be fairly the same delay on other files, regardless of the size of the file.

Now, for the file with 25,000 records:

25,000 records

Using CFFIle:
Start:{ts '2006-11-03 10:47:46'}
End:{ts '2006-11-03 10:47:55'}

Using Java FileReader
Start:{ts '2006-11-03 10:47:55'}
End:{ts '2006-11-03 10:47:59'}

After 25,000 records, the CFFile took 9 seconds, and the Reader objects took 4 seconds. That is quite a difference. So, we ended up using the Java code, and it works well. We did some other mods, like using the <cftry> and <cfcatch> blocks to handle errors gracefully. If this was a full Java approach, I would also use the finally {} block to close the Reader objects. The moral of the story is, we have options.

More about Tomcat Contexts

Here is some more relevent information about Contexts in Tomcat. This is just an addition to the post I made yesterday. I found this on the Tomcat Deployer How-To webpage.

A Word About Contexts

In talking about deployment of web applications, the concept of a Context is required to be understood. A Context is what Tomcat calls a web application.

In order to configure a Context within Tomcat a Context Descriptor is required. A Context Descriptor is simply an XML file that contains Tomcat related configuration for a Context, e.g naming resources or session manager configuration. In earlier versions of Tomcat the content of a Context Descriptor configuration was often stored within Tomcat's primary configuration file server.xml but this is now discouraged (although it currently still works).

Context Descriptors not only help Tomcat to know how to configure Contexts but other tools such as the Tomcat Manager and TDC often use these Context Descriptors to perform their roles properly.

The locations for Context Descriptors are;

1. $CATALINA_HOME/conf/[enginename]/[hostname]/context.xml
2. $CATALINA_HOME/webapps/[webappname]/META-INF/context.xml

If a Context Descriptor is not provided for a Context, Tomcat automatically creates one and places it in (1) with a filename of [webappname].xml although if manually created, the filename need not match the web application name as Tomcat is concerned only with the Context configuration contained within the Context Descriptor file(s).

Tomcat JSP and Servlet Reloading

Well, for my first official blog post, I'd like to talk about a problem I've recently ran into with JSP loading in Tomcat 5.5. Now, I've used Tomcat for years, but never took the time to learn it totally inside and out. I've become very good at using Tomcat over the years by tackling problems as they arise. So, I am very comfortable with it.

So, last week, when I uploaded a JSP to my Tomcat web application directory, and it didn't show the changes I've made to it, I was perplexed. This was the most cut and dry part of using Tomcat. Uploading JSPs and having them dynamically reload. Why was this happening? Why? Why? Why????? After researching the problem, I found that others had the same problem, but their questions were never answered, or the solutions were not available on the web. So hopefully, this post will help someone.

First, I will describe the environment. This is a server that was created for students, at the university that I teach at, to use for assignments. It is an Apache Tomcat version 5.5.17 application server, on a Sun Sparc OS 5.10. Since this is an intro to dynamic web development class, the students do not deploy WAR files. Rather, they deploy their servlets and JSPs to a pre-created web application directory, under the tomcat "webapps" directory. When I created this directory, I did not change any Tomcat configuration files. I just created the root directory "classfiles", and the WEB-INF, classes, and lib directories.

The first problem was loading servlets. That one was an easy one. The web.xml file needs to tell Tomcat what to look for and how to load servlets. So, I made a basic web.xml file under the directory "classfiles/WEB-INF" that included the following code in it:

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE web-app
PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
"http://java.sun.com/dtd/web-app_2_3.dtd">
<web-app>
<servlet>
<servlet-name>invoker</servlet-name>
<servlet-class>
   org.apache.catalina.servlets.InvokerServlet
</servlet-class>
<init-param>
   <param-name>debug</param-name>
   <param-value>0</param-value>
</init-param>
<load-on-startup>1</load-on-startup>
</servlet>
<servlet-mapping>
<servlet-name>invoker</servlet-name>
<url-pattern>/servlet/*</url-pattern>
</servlet-mapping>
</web-app>

This tells tomcat to look for anything that has the /servlet/ in the user for the "classfiles" application and load it as a Servlet. For example: http://localhost:8080/classfiles/servlet/HelloWorldServlet

Easy enough, however, the dynamic reloading of the servlets wasn't happening. After looking through the documentation on the Tomcat site, I found out that Tomcat has to be explicitly notified about web applications, and if they should be reloadable. By default, if I was to use WAR file deployment, Tomcat would handle this automatically. Since this is not the case, I need to explicitly tell Tomcat what to do. This can be done either by modifying the server.xml file, in the tomcat "conf" directory. Or, to give me more direct control over the web application, and reduce the risk of accidentally screwing up someone else's context, I decided to make my own context configuration. This is done by creating a new directory under your web application directory called "META-INF". Then creating a file in there called "context.xml". Therefore, the contents of my file at "classfiles/META-INF/context.xml" was as follows:

<?xml version='1.0' encoding='utf-8'?>
<Context displayName="Software Engineering Student Projects"
reloadable="true">
</Context>

I am telling Tomcat to monitor the WEB-INF/classes and WEB-INF/lib directories for any changes. And WHAM, the servlets, packages, and other classes are now reloadable. But what about JSPs??? Am I missing something? Aren't JSPs just simple servlets? Well, yeah, sort of. It turns out that you need to set the docBase for the JSPs to reload. So now my context.xml file looks like this:

<?xml version='1.0' encoding='utf-8'?>
<Context displayName="Software Engineering Student Projects"
docBase="/usr/local/tomcat/classfiles" reloadable="true">
</Context>

And now my JSPs are reloading with no problems.

So the context.xml file can be used to make your pages, servlets, classes, and packages reloadable, if you are not using the default deployment capabilities of Tomcat. Tomcat does all this stuff automatically when deploying applications as a Web ARchive, WAR file. Is that it??

"But that is not all I can do" said the cat. Through research into the context.xml, and server.xml, files, you will find a ton of more capabilities that these files provide. Such as data sources, cross context sessions, and application level variables. But that is a post for another time.

For more information, check out the Tomcat documentation on configuring the Context container.