Showing posts with label Java. Show all posts
Showing posts with label Java. Show all posts

Wednesday, May 18, 2011

Ruby on Rails vs Java ORM

Wow, I am having fun with Ruby on Rails. The migrations and scaffolding alone make me less of a coder and more of an Engineer. Lol, actually, it is kind of like that movie Wall-E. You know... where all the humans relied on the robots to do everything, evolving the humans into weak boned blobs. Rails does a lot for you, and it is hard for a Java programmer, like myself, to give up the deep control I had over the software. Or is it???

One of the concepts I learned today was the use of named scopes in Rails. The concept is that you can predefine a db query, which can be dynamically added to any normal method call on the Model. For example, lets say you have 2 tables, Customer, and Address. A Customer can have one or many Address records. So, I want to capture that in my Customer model. The default functionality for the Customer model would be to return one, or many, Customer records, using the find() method. Which is good, but I also want to return the Address information as well. Consider the following Customer model:
class Customer < ActiveRecord::Base
has_many :addresses, :dependent > :delete_all, :foreign_key => 'person_id'
named_scope :with_address_info, {
:select => "customers.*, addresses.*",
:joins => "JOIN addresses on addresses.person_id = customers.id",
:order => "customers.last_name asc" }
end
The above code is all I need for a simple Customer model, customer.rb. The has_many identifier tells Ruby that the Customer can have many Address records. The "with_address_info" named scope allows me to dynamically include the Address records with any query. In my Controller, I can have the following code:
customers = Customer.find(:all)
customerFull = Customer.with_address_info.find(:all)
That's all. Pretty powerful stuff. That small model file has given me the ability to Create, Retrieve, Update, and Delete (CRUD), all in 7 lines of code. Technically, if I don't care about the named scope, the size of the file would be 3 lines of code. But that seems pretty scary. It is like magic. Obviously, it isn't magic. The Rails framework writes all of those extra lines of code for you behind the scenes.

So, does it really take away control from a programmer, more than Java? Well, if you are use to manually creating JDBC calls to a database, then YES. You have a lot more control with the DB access in the code itself when you work with the straight JDBC drivers.
But, the need for that level of control is rare. Many of us use Object Relational Mapping (ORM), like Hibernate or Java Persistence API (JPA). In that case, Java takes the JDBC overhead away as well as the Rails framework. Of course, the syntax between Ruby and Java is such that a Ruby Model is much smaller in lines of code then a JPA entity (all those getter and setters add up). There is also the dynamic typing that I won't get into. But if you consider the "magic" with mapping a software object to a database object, then they are performing the same actions. Consider the following JPA Entity:

import java.io.Serializable;
import javax.persistence.*;
import java.util.Set;
@Entity
@Table(name="customers")
public class Customer implements Serializable {
@Id
private int id;
@Column(name="customer_type")
private String customerType;
@Column(name="email_address")
private String emailAddress;
@Column(name="first_name")
private String firstName;
@Column(name="last_name")
private String lastName;
@OneToMany(mappedBy="customer")
private Set<Address> addresses;
public Customer() {
}
public int getId() {
return this.id;
}
public void setId(int id) {
this.id = id;
}
public String getCustomerType() {
return this.customerType;
}
public void setCustomerType(String customerType) {
this.customerType = customerType;
}
public String getEmailAddress() {
return this.emailAddress;
}
public void setEmailAddress(String emailAddress) {
this.emailAddress = emailAddress;
}
public String getFirstName() {
return this.firstName;
}
public void setFirstName(String firstName) {
this.firstName = firstName;
}
public String getLastName() {
return this.lastName;
}
public void setLastName(String lastName) {
this.lastName = lastName;
}
public Set<Address> getAddresses() {
return this.addresses;
}
public void setAddresses(Set<Address> addresses) {
this.addresses = addresses;
}
}
The Customer entity is a lot more wordy then the Customer Rails Model, but it performs the same kind of magic, just a little differently. If you were to use this Entity in a session EJB, you could have the following code:
List customers = entityManager.createQuery(" from Customers c ");
This code already has a reference to the address records. You can just call the getAddresses() method:
for (Customer customer : customers) {
List<Address> addresses = customer.getAddresses();
....
}
So, overall, Ruby on Rails provides a quick, and less verbose method of getting data across multiple tables. But if you use some popular ORM packages, you really arn't losing the control you thought you had.

Thursday, December 30, 2010

Java EE Packaging

During my off hours, I do a little teaching. The course I teach relates to Java EE and EJB development. Over the past 3 years that I've been teaching this course, I've realized there are quite a few common issues that students get hung up on. Initially, the issues with classpaths, packaging, and deployment are the biggest hurdles. So, here is an overview about the packaging of Java components. I just JBoss as an example, but most Java EE servers are handled in a similar way.

First, we have the Java ARchive, or jar, file. A jar file is one of the earliest package formats in Java, and can be used to compress, and package classes together in a neat library. In the EJB world, the jar is what you use to package your EJBs. You can package EJBs by themselves, and deploy them separately, or they can be included as an Enterprise application (more on that later). The jar file contains your classes, and a META-INF directory. The META-INF directory contains config and deployment descriptor files. To create a JAR file, just go to the top level of your class hierarchy and type "jar cvf jarFile.jar .". This will package your classes, properties, config, and deployment descriptor files into a single jar file. From here, you can deploy it straight to the Java EE server. It should be noted that some Java EE servers require some extra deployment descriptor files, refer to the vendor documentation for more information. However, in the case of JBoss, you can just copy the file into the "deploy" directory, and JBoss will run with it.

Upon deployment, JBoss will unpack the code, and set up the EJBs to be used. During the deployment, errors can happen. This is a good time to pay attention to the logs. Many problems happen with the deployment descriptors, or when you are using injection, with annotations. So, looking at the logs will help you to determine if the problems are runtime, or deployment time. A successful deployment would contain something similar to the following in your server.log, or standard output.
2010-12-29 10:01:08,666 INFO [org.jboss.ejb3.session.SessionSpecContainer] (HDScanner) Starting jar=MyTestEjbs.jar,name=HelloWorld,service=EJB3
2010-12-29 10:01:08,677 INFO [org.jboss.ejb3.EJBContainer] (HDScanner) STARTED EJB: awg.ejb.session.HelloWorldImpl ejbName: HelloWorld
2010-12-29 10:01:08,752 INFO [org.jboss.ejb3.proxy.impl.jndiregistrar.JndiSessionRegistrarBase] (HDScanner) Binding the following Entries in Global JNDI:
HelloWorld/remote - EJB3.x Default Remote Business Interface
HelloWorld/remote-awg.ejb.session.HelloWorldRemote - EJB3.x Remote Business Interface
The above log entry not only tells me that the ejb was deployed without problems, but it also give me some information about the ejb. For example, the JNDI name. In this case, the JNDI name is "HelloWorld/remote". This is very useful for when you are writing your client code.

Once the EJB is deployed successfully, you can use it in some client code. You can write a local client, or you can create a web based client. The important thing to remember when writing a client to access an ejb is that ejbs are considered remote objects. So, you need to supply some information in your code on where to find the references to the ejbs. This is commonly performed using the JNDI. If you are accessing the ejbs from a jsp, and it is deployed on the same server instance, then the access to the ejb is pretty simple. Just get a reference to the Initial Context of the JNDI, and pass in a String, for the name of the resource you are trying to get. For example, if we want a reference to the HelloWorld ejb I deployed, I would use the following code:
InitialContext ctx = new InitialContext();
awg.ejb.session.HelloWorldRemote hwr = (awg.ejb.session.HelloWorldRemote) ctx.lookup("HelloWorld/remote");
However, if we are using a client remotely, we need to write code that is more involved. For example, consider the following command line client:
package awg.client;

import java.util.Properties;
import javax.naming.Context;
import javax.naming.InitialContext;
import awg.ejb.session.HelloWorldRemote;

public class CommandLineHelloEJBClient {

 public static void main(String[] args) {
   try {
     // create an InitialContext, with JNDI properties
     Properties p = new Properties();
     p.put(Context.INITIAL_CONTEXT_FACTORY, "org.jnp.interfaces.NamingContextFactory");
     p.put(Context.PROVIDER_URL, "jnp://localhost:1099");
     Context ic = new InitialContext(p);

     // Get the a reference to the Remote Object from JNDI
     HelloWorldRemote client = (HelloWorldRemote)ic.lookup("HelloWorld/remote");

     // run methods on the ejb
     client.....();

   } catch (Exception e) {
     e.printStackTrace();
   }
  }
}
In the above example, I am giving the URL, jnp://localhost:1099, and the Context type.

With local clients, we can wrap them in jar files as well and run them from the local command line. But what about web resources, like JSPs? In the Java world, web resources can be packaged up in their own package as well. The Web ARchive, or war file. The WAR follows a specific directory structure to allow for easy deployment of web resources.

The top level directory is considered the webroot, and it contains JSPs, HTML, CSS, JavaScript files, images, and any other file you want to be exposed to the web. Within the webroot, there is also a WEB-INF directory. Inside the WEB-INF directory, config files, and deployment descriptors, are kept. Also in the WEB-INF directory, there is a directory for your classes, and a lib directory, for any libraries your'd like to package with your web application. To create a war file, just go to the webroot directory and type "jar cvf warFile.war .". This will package your web files, and additional classes and jars into a war file. From here, you can deploy it your Java EE server, just like you did with the ejb jar file.

The jar and war files are technically all you need to run your web files, and ejbs. But to make the deploying and submitting even easier, and cleaner, you can use an Enterprise ARchive, or ear, file.
The ear file is nothing more than a single package containing your war file(s), jar file(s), and deployment descriptors. To build an ear file, follow these steps:
  1. Package up your ejbs into a jar file
  2. Package up your web resources into a war file
  3. Throw your war and jar files into a directory where you can package them together
  4. Create a new directory called META-INF (under the directory containing your packages)
  5. In the META-INF directory, create a deployment descriptor called application.xml.
    * The application.xml file defines the modules in your application.
  6. From the directory containing your packages, package everything up as an ear file: jar cvf MyApplication.ear .
  7. Deploy the ear to the JBoss deploy directory.
Seems like a lot of work, however, you can build Ant script, or use an IDE like Eclipse or Net Beans to do all of the heavy lifting. I'll talk more about that in another post.

The ear file makes deployment much more organized, and testing of the module is simpler. One thing to remember is that the Context lookup will be different with an ear, compared to using a war and jar separately. The ear context is basically the name of your ear file. For your lookup, you need to reference the ear context, when using JBoss. For example:
  • I created an ear called MyApplication.ear. The file contains all of the examples from the class. changed all of the lookups to include "MyApplication/" at the beginning of the look up. If I deployed the war and jar files separate, the code in the HelloWorld client would look like this:
    HelloWorldRemote hwr = (HelloWorldRemote) ctx.lookup("HelloWorld/remote");
  • However, since I am deploying it as an ear file, the code now references the ear context:
    HelloWorldRemote hwr = (HelloWorldRemote) ctx.lookup("MyApplication/HelloWorld/remote");
It should be noted that the JNDI names are not standard. Each Java EE server has it's own way for referencing ejbs on the JNDI. You may need to refer to documentation to get the format of the name. You can also use the ejb-jar.xml file in the META-INF directory of you ejb package, to customize the name, but there are still vendor specific formats tied in. That is why they are called by string names. If you write your code correctly, you can hold the names in a properties file, XML, or even a database. This will keep your code more portable. However, in EJB 3.1, JNDI names will be global and will follow a standard pattern that will make your code more portable across different application servers. But that is another story.

For more information, you can refer to the JBoss "Getting Started" documentation: https://www.jboss.org/file-access/default/members/jbossas/freezone/docs/Getting_Started_Guide/beta500/html-single/index.html#Configuration_Files

Monday, December 27, 2010

Installing & Running JBoss in Ubuntu

Being mostly a Java guy, a couple of my favorite servers to work with are Tomcat and JBoss. While at work, it has been exclusively Tomcat, lately, I've been mostly interested in working with JBoss. For a couple years, I've had JBoss installed on my Windows machine. It is pretty easy,just unzip on your hard drive and execute the run script in the bin directory.

Since I installed Ubuntu, I've wanted to switch to using Linux as my JBoss OS. With Ubuntu, I could easily use the update manager, or apt-get jboss, to get it installed. However, it wouldn't give me the version I wanted. So, I figured I'd work this manually.

The first thing I do, is download a zip. A common place to put the software is /usr/local. So, I will download, and unzip the package there.
cd /usr/local
sudo wget http://sourceforge.net/projects/jboss/files/JBoss/JBoss-6.0.0.Final/jboss-as-distribution-6.0.0.Final.zip
sudo unzip jboss-as-distribution-6.0.0.Final.zip
The funny thing is that I can't seem to find a tarball for JBoss, so I just get the zip. Based on this, and the directory structure of JBoss, it seems that the development of JBoss was mostly geared towards Windows based servers. Regardless, it works great on Linux as well.
Now, at this point, you can just run JBoss out of the box, using the run.sh script in th ebin directory.
sudo /usr/local/jboss-6.0.0.Final/bin/run.sh
This will start up the server, and the terminal window would be the standard output. It that is all you need, then you can stop right here. But in general, there are a couple things to simplify the start up and shut down process. I like to control the server as a service, from the /etc/init.d directory. Within the bin directory, there is a startup script which will call the run.sh script as the jboss user. The script is written with redhat in mind, but is easily modified to run with Ubuntu. But first, lets create a jboss user and grant ownership to the jboss directory.
sudo useradd -s /bin/bash -d /home/jboss -p jboss
sudo chown -R jboss:jboss /usr/local/jboss-6.0.0.Final/
The useradd command will create the jboss user, with the /home/jboss directory as it's home, and the default shell is bash. This command will create the jboss user without a password. If you are like me, and like to make things as secure as possible, use sudo passwd jboss to create a password for the jboss user.

sudo ln -s jboss-6.0.0.Final/ jboss. The chown command grants ownership to the jboss user. The -R, after the chown command, does a recursive call to the sub directories.
Now that the jboss user is set up, copy the jboss_init_redhat.sh file into the /ect/init.d directory, and name it jboss.
sudo cp jboss-6.0.0.Final/bin/jboss_init_redhat.sh /etc/init.d/jboss
There are a couple changes we need to make with the file. first, it assumes that the JBoss home directory is /usr/local/jboss. By default, our command to unzip the package created the home as jboss-6.0.0.Final/, so we can either rename the directory, with the mv command, change the init file to reflect the real path, or create a symbolic link. I prefer the symbolic link, because we can keep the old packages as we download more, and if we need to revert, it is just a matter of modifying the symlink.
sudo ln -s jboss-6.0.0.Final ./jboss
The next change is in the file itself. Use VI and modify the JAVAPTH variable to the correct path of your java command. In my case, and most likely yours as well, the path is /usr/local/bin. If you don't know the path, just use the command "which java". Verify some of your other variables, just in case you need to make changes for your system.

At this point, you are all set to run it as a service. Use the following commands to start, stop, and restart the server.
/etc/init.d/jboss start
/etc/init.d/jboss stop
/etc/init.d/jboss restart
If you created your jboss user with a password, you will need to supply the password. The server will start up and the logs will be written to /usr/local/jboss-6.0.0.Final/server/default/log/server.log. If you'd like to have the server start at system start, then you can update the rc.d to include it as an auto start up.
sudo update-rc.d jboss defaults
I personally prefer to start it up as needed. Once started, you can deploy your jars, wars, and ears as you normally would, in the deploy directory of the server you are using. In this case, we are using the default server.

Update 12/29: I updated this post to reflect the GA release of JBoss 6.0.

Friday, October 22, 2010

Software Engineer Job Interviews

About 2 months ago, I was notified by a friend, about some job openings for Software Engineers, at her company.  I am pretty comfortable in my current position, but this other company looks pretty cool. Things like flex time, no dress code, high tech environment, and free coffee, makes me think that it could be the perfect job.

After a couple hours of research, it also looks like this company has high standards and employs really talented people. Not just a “Web Development” company, they expect you to understand Big O notation, and different design patterns. I consider myself an “Average Web Geek”, but I did receive a Masters in Computer Science, and my interests have always leaned more towards the engineering of software, compared to the assembly line culture that some IT companies represent.

I talked to the recruiter of this company, pretty standard stuff. She tells me that there are 4 phases to the interview process. There is first contact, that is that talk with the recruiting officer. Then there is a “homework” assignment, so the engineers can evaluate my initial coding skills. Then there is a phone interview with an engineer. Finally, there is a face-to-face. With the exception of the “homework” it is pretty standard.

Now, I should explain, I enjoy my job. But I am frustrated by the lack of a technical track. My company is looking for people to build web sites, then spend the rest of their professional lives as non-technical managers. I am also realizing there is a significant amount of co-workers who do not take as much pride as I do in engineering, and innovation. I consider myself the “Average” web geek, because I don’t see myself as anything special. Though, when I look around my office, I see a lot of people who are comfortable with their position, and the fact that they have no need to advance their knowledge, or experience. So, as happy as I am, I need to expand, and continue to advance my technical experience. This other company looks to have the direction I want to focus on.

The recruiter sends me the assignment. There are a couple questions about object oriented programming, and a Java interface for which I am required to supply an implementation. Question 1 is “What is an abstract class? Describe a common pattern that requires the use of abstract classes?” This is interesting. Normally I would use this as a question in a phone interview. It is a pretty basic question to test someone's basic object oriented knowledge. The other questions are specific to Java.

Based on the Job description, I know that they want an engineer with a large amount of Java experience, along with Spring, Hibernate, and JUnit. So, even though it isn’t required, I make sure that the code I send with the implementation class, contains these these concepts.

After a couple weeks, I hear back from them. They want to to set up a phone interview with a Lead Engineer. At this point, I am pretty excited. They seem like a pretty high tech company, with some really smart employees. So, I agree with excitement. The call comes in a couple days later. I take the call in the parking lot of a Trader Joes. I was expecting the interviewer to ask about my Java experience, shoot me some questions about Spring and Hibernate. Maybe test my knowledge of Java core concepts. But no, he hits me with a question to evaluate how I think.

He says “Lets say you are designing a card game, and you need a method to shuffle the cards. Describe the algorithm to do it.”

This question is very familiar, since I had it as a test question in my algorithms class during my graduate studies.  Unfortunately, that was about 8 years ago.  Though it shouldn’t be too hard. So, my first answer is to use 2 arrays. 1 contains 52 Card objects, and the other is empty. Then, using a random number generator, I would grab cards at random, and store them in the other array. Now, my current job hardly requires me to consider algorithms which are really speedy. Our datasets are small, and the schedules are short. So, usually the first solution is fine.  So, I was expecting his next question… “What about the blank cells in the original array? Are you concerned about those?”

Yes, of course. So, without thinking about redesigning the algorithm, I tell him that each time we remove a Card object, we can recreate the array with the size decremented. As I am talking I am realizing how stupid, and inefficient this sounds.  So I say, “It will work, but it isn’t efficient.” He then asks me to describe the performance in Big O notation. So…. there are n Card objects in the array (n = 52) , looping through the array n times to remove the Card objects. Then I would recreate the array as n-1, for each card. That is n*(n-1). Making the performance O(n^2), aka quadratic. Not very good considering that there is a better way.

After a couple minutes, it comes to me. There is actually a much simpler solution. I don’t need 2 arrays. Using just the original array of Card objects, I can generate 2 random numbers, and swap the cards. The random number generator, and swap function, would run in constant time, O(1). That leave the number of iterations. Therefore, the performance is O(n), aka linear. Much better.

The following code represents this algorithm:

public static void shuffleCards(Card[] deck) {
  Random generator = new Random();
   // shuffle the cards
   for (int i = 0 ; i < deck.length ; i ++) {
       int x1 = generator.nextInt(n) + 1;
       int x2 = generator.nextInt(n) + 1;
       Card card1 = deck[x1];
       Card card2 = deck[x2];
       deck[x1] = card2;
       deck[x2] = card1;
   }
}

This was the answer he was looking for. At this point, I could have pointed out that the code could be simplified by using the java.util.Collections class, which has a shuffle(List<?> list) method, and runs in linear time.

The phone interview continues with me asking questions about companies, and him asking a couple more questions about my experience. I guess I made an impression, since the recruiter schedules a series of face-to-face interviews. That will happen in the next week. So we would see what happens.

The moral of the story is, that there is a difference between being a average web coder, and a software engineer. I want to see what else is out there, so I have to break out of my IT shell and focus on the foundations which I learned in those Computer Science courses. It is very easy to grow comfortable and content in a stable position. But time keeps moving, and if you don’t keep learning, you can find yourself stuck. I hope one day I’ll consider myself to be more then just an Average Web Geek.

Friday, May 23, 2008

AD Authentication and Java

Well, it was only a matter of time before my job would require our Java apps to authenticate against Active Directory. For those who don't know what Active Directory (AD) is (myself included up till last year), AD is a Microsoft Windows implementation of LDAP. It is typically used in Windows 2000 based networks to tie together the standard IT resources like mail, calendars, and desktop computers. Those of us who are on a network that uses AD typically have a desktop that authenticates against AD, as well as MS Outlook for email. The good thing about a central authentication source for network credentials is that is allows for a single username and password to be used for things. Which can eventually lead to "Single Sign-On" (which will be discussed later, when I learn how to do it).

Anyway, a central source for authentication credentials also helps with web applications on that network, because now we no longer need to worry about password management, or user registration. That can reduce the project schedule by a week or two, depending on how strict your organization's password and user registration requirements are.

So, how do we do it? Well, it is actually pretty simple. Like I said, AD is basically another version of LDAP, so in Java you can use the Java Naming and Directory Interface (JNDI). There are a bunch of LDAP classes in the javax.naming.ldap package that can help. And because the Java API is so robust, it gives you a ton of flexibility to customize your code as much as possible. At the same time, it can seem a bit intimidating. Sun has some pretty good information on their website about LDAP authentication, which can also be used for AD Authentication. Lets take a look at some code.
Hashtable env = null;
DirContext ctx = null;
boolean isAuthenticated = false;

try {
try {
String loginId = "yourdomain\\avgwebgeek";
env = new Hashtable();   // hash table for your LDAP properties

// set up the LDAP properties
env.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory");
env.put(Context.PROVIDER_URL, "ldap://myadserver.mydomain.com:389");

// Set the authentication mechanism to be simple
env.put(Context.SECURITY_AUTHENTICATION, "simple");

// login credentials
env.put(Context.SECURITY_PRINCIPAL, loginId);
env.put(Context.SECURITY_CREDENTIALS, password);

// the following is helpful in debugging errors from the
// AD server side of things
//env.put("com.sun.jndi.ldap.trace.ber", System.err);

// Create the initial directory context
ctx = new InitialDirContext(env);
isAuthenticated = true;
} catch (AuthenticationException e) {
// this exception is typically found with incorrect credentials
e.printStackTrace();
} catch (NamingException e) {
e.printStackTrace();
}
} catch (Exception e) {
e.printStackTrace();
} finally {  
try {
ctx.close();
}catch (NamingException e) {
e.printStackTrace();
}
}

if (isAuthenticated) {
System.out.println("I am authenticated.");
} else {
System.out.println("I am not authenticated.");
}

This is a very simple application. It does the job, however, it is not very secure. The problem is that I am setting the Context.SECURITY_AUTHENTICATION to "simple". What this means is that your password is being sent over the network through clear text. If someone is running a network sniffer, they can read your password from your IP packets. One way around this is to make sure that your LDAP connection is handled over SSL. In other words, use "ldaps://" with port 636, and not "ldap://" on port 389.

If your server is not setup to handle SSL, or you just want a little extra security, you can change the Context.SECURITY_AUTHENTICATION to "DIGEST-MD5". However, as I found out, AD treats this a little differently. The concept of using "yourdomain\\yourusername" needs to be reduced to just "yourusername". It has something to do with how AD 2003 sets up the HASH for the MD5. This method will use a Hashing algorithm to be verified with the server, that you indeed know your password.

So, that's it for AD authentication. Like I said, the Java API can let you do a lot more. You can set up controls to do searches, find specific user information, and find groups assigned to users. AD groups can be used to help with the Authorization portion of your apps. Take a look at the javax.naming.ldap API, there is a lot there, but it isn't too hard to follow.

Friday, November 03, 2006

Reading Files With ColdFusion, or Java???

One project I was on, not to long ago, consisted of transferring a datafile between 2 systems. Unfortunately, this datafile wasn't XML. It was a pipped delimited file of product records. The main purpose of this file was to sync up data between the two systems. Therefore, I needed to read the file, line by line, parse out each field of the file, and compare it with current records in the database. Since the language of choice for this particular project was ColdFusion, I was encouraged to keep with the standard.

Being a Java guy, with some php and perl background, I knew I could adapt. But how can I make this as efficient as possible. The app server we were using was ColdFusion version 6.1. So, I know I had options. At least it wasn't version 5 or below, like some other organizations who shall remain nameless.

Anyway, ColdFusion has a tag called <cffile>. This tag is nice and easy to use. However, it has one main drawback to it. It holds the entire file in memory at once. It might not seem like a problem, but I am talking about somewhere around 200,000 records. With each record containing 80 fields. That can make the application more prone to memory errors. Especially, if this is a user requested job, and not a scheduled job.

However, the standard Java development kit has the FileReader, used with a BufferedReader, that can stream the file, line by line. So, I figured I'd give it a try. I started by comparing the two methods of file reading. I wrote a sample ColdFusion page which would read the file in and compare the time it takes to read/process the file using the <CFFile>, and the Java FileReader methods. I started with a small file, containing 200 records. Then moved up to a file with 25,000 records. The code is as follows:
Using cffile:
<cfset filePath = "./DataFile200Records.txt">
<cffile action="read" file="#filePath#" variable="fileContents">

Using CFFile<br>
<cfoutput>
Start:#now()#<br>
<cfloop index="line" list="#fileContents#" delimiters="#Chr(10)#">
<!--- Account for empty values --->
<cfset newline = " " & Replace(line, "||", " | | ", "all") & " ">
<!--- Convert the pipped delimited list to an Array --->
<cfset fieldArray = listtoarray(newline,"|")>
<cfloop index="i" from="1" to="#arraylen(fieldArray)#">
<!--- Loop through each field --->
<!-- #fieldArray[i]# *** -->
</cfloop>
</cfloop>
End:#now()#<br> <!--- End Timestamp --->
</cfoutput>

Using Java FileReader:
Start:<cfoutput >#now()#</cfoutput><br><!--- Start Timestamp --->
<cfscript>
// create a FileReader and BufferedReader objects
fileReader = createObject("java","java.io.FileReader");
buffReader = createObject("java","java.io.BufferedReader");

// Instantiate the FileReader with the file path
fileReader.init(filePath);

// Instantiate the BufferedReader with the fileReader Object
buffReader.init(fileReader);

// read the first line into a String
fileLine = buffReader.readLine();

// Loop while the String is defined
while (isDefined("fileLine")) {
// Account for empty values
newline = " " & Replace(fileLine, "||", " | | ", "all") &amp;amp;amp;amp; " ";
// Convert the pipped delimited list to an Array
fieldArray = listtoarray(newline,"|");
for (i = 1; i lt arraylen(fieldArray); i = i+1)
{
// Loop through each field
writeOutput("<!--" & fieldArray[i] & " *** -->");
}
// read the next line to continue the loop
fileLine = buffReader.readLine();
}

// close the Reader objects
buffReader.close();
fileReader.close();
</cfscript>
End:<cfoutput >#now()#</cfoutput><!--- End Timestamp --->
Essentially, each section of code is doing the same thing. It reads the file, line by line. Parses the fields, including empty fields. And it loops through each field, displaying it as an HTML comment. The main difference is that the Java FileReader and BufferedReader objects have to be closed when they are no longer needed. You always want to close a Streamed object, otherwise it can take up memory.

So, what are the results? You would think that CFFile would run faster, being that the file is in memory. And memory access is usually pretty quick. Well, for the small, 200 record, file, that is correct. Below are the results:
200 RecordsUsing CFFile:
Start:{ts '2006-11-03 10:23:32'}
End:{ts '2006-11-03 10:23:33'}

Using Java FileReader
Start:{ts '2006-11-03 10:23:33'}
End:{ts '2006-11-03 10:23:35'}

Using the Java objects took a second longer. I think that may have to do with the overhead involved with opening and closing the Reader objects. But that would be fairly the same delay on other files, regardless of the size of the file.

Now, for the file with 25,000 records:
25,000 recordsUsing CFFIle:
Start:{ts '2006-11-03 10:47:46'}
End:{ts '2006-11-03 10:47:55'}

Using Java FileReader
Start:{ts '2006-11-03 10:47:55'}
End:{ts '2006-11-03 10:47:59'}

After 25,000 records, the CFFile took 9 seconds, and the Reader objects took 4 seconds. That is quite a difference. So, we ended up using the Java code, and it works well. We did some other mods, like using the <cftry> and <cfcatch> blocks to handle errors gracefully. If this was a full Java approach, I would also use the finally {} block to close the Reader objects. The moral of the story is, we have options.