Whether your district has six systems or 60, having a coherent plan to connect them is absolutely essential. Here, an overview of your technical options.
Question (multiple choice, 15 points): A school CIO has 15 mission-critical systems, each of which must be integrated with other programs for reporting and data-sharing. Five of those systems are currently being replaced; another five are being upgraded; two are being consolidated into one new system; and three new systems are being added. Given a general shortage of resources, increasing pressure to integrate multiple data sources, a plethora of complex proprietary systems that don’t adhere to industry standards, and the constantly accelerating pace of change in technology, what does the CIO have?
a) (n2-n)/2
b) An enormous headache
c) No Child Left Behind
d) A desperate need for an application integration strategyThe answer, of course, is e, all of the above, but let’s start with d. IT systems need to be able to communicate for a wide variety of reasons, including reporting, data warehousing and aggregation, and elimination of duplicate data and processes. There are two primary approaches to link up diverse systems. The first is to build custom interfaces directly between each set of systems you need to integrate. This could take the form of a piece of custom code, a data export or report, or at the lowest common denominator, a human being doing the old cut-and-paste or data rekeying between applications. The second—and much more promising— method is to deploy an enterprise integration system (also referred to as enterprise application integration). With a dedicated EI system, instead of writing a custom interface, you purchase or write a single interface for each enterprise system, and that interface plugs directly into the central EI system.
As CIO, you may not be in charge of implementing technical architecture, but no doubt you’ll have to oversee the overall strategy—and understand the costs and benefits of the different approaches. Which route is right for your district? Read on for a map to your integration options.
GOING THE CUSTOM ROAD
Building a custom interface is more common than using a centralized EI system, and cheaper in the short run, since each interface can be built on an as-needed basis, using whatever methods and monies are available at the time. Your student information system needs to “talk” to your food service system? Just build an interface. But the ongoing upkeep and maintenance costs for custom interfaces can be crushing, and they increase exponentially. Here’s the math: in an environment with n systems, the total number of interfaces between those systems, assuming that every system needs to “talk” to every other, is (n2-n)/2. In other words, if you have three systems to connect, you have three interfaces— not so bad. But if you have 10 systems to integrate, the number of interfaces is 45. And if you have 20 systems, the number of potential interfaces is 190. And it gets worse. Since these interfaces were likely built in an ad-hoc fashion over a long period of time, they’re probably written in several computer languages by a variety of staff, using different tools and technologies that may be incompatible with one another. Every time you change one of your IT systems, you’ll have to analyze and possibly rewrite every interface. Nevertheless, if you have a small number of systems to connect, or simply lack the resources to invest in a dedicated EI solution, custom interfaces are your answer. Here are the different methods to consider.
APPLICATION PROGRAM INTERFACES
One way school IT staff can connect different systems is by writing custom code using one or more API standards. APIs are usually vendor-created mechanisms designed to expose some specific functionality in their products, so programmers can access them and create new functions and features. A new variant on APIs consists of vendorneutral Web services standards designed to expose specific functions to other Web services-aware applications.
Pro: Accessing a system via its API is sometimes the cleanest way to get to its functions and data, since you’re using a method designed by the system vendor. Also, writing interfaces based on the published APIs of programs means you’re unlikely to accidentally “break” the underlying programs.
Con: You can usually do only what the vendor who wrote the API allows you to do, and you need skilled staff able to program and maintain code. Bottom Line: This is often your best option for writing custom interfaces if your products have good APIs to work with.
FULL ACCESS
An extension of the API method of interface creation is having full access to the source code of the systems being integrated. Typically, this occurs only when a system or systems were developed in-house, where a vendor offered source code as a part of the purchase, or where open source systems have been adopted.
Pro: You have complete freedom to integrate systems by extending their functionality…
Con: …assuming that you have the programming resources and expertise available. Bottom Line: It’s nice if you have this option, but usually you don’t.
SINGLE-VENDOR SOLUTION
Another solution is to buy all of your IT systems as an integrated bundle from a single vendor.
Pro: The hope is that you’d get clean, out-of-the-box integration between components.
Con: Despite claims to the contrary, no one vendor will ever have a solution that will meet all of your current and future needs. Aside from this, vendor products that are internally integrated are often less open to outside integration with third-party products and solutions. The logic is, “if we give it all to you, why would you ever want anything else?”
Bottom Line: Unless your needs are simple, it’s usually wise to resist the sales pitch and avoid getting locked into a single-vendor solution.
DIRECT DATA-LEVEL CONNECTIVITY
Data-level connectivity means digging down to the data source underlying the system or systems you want to connect. This approach is used frequently when a district has bought only the compiled code of a student information system that doesn’t have an API, or the API provided is inadequate. Following this route requires either having access to the database schemas behind your systems, or reverse-engineering them by figuring out what each data field represents and how it is used. A common variation on this method is to pass data indirectly from program to program using file exports and imports.
Pro: This approach allows you to bypass the code of the system, which is important if you don’t have access to the source code or to the right APIs.
Con: By circumventing the program code and accessing the database directly, you risk bypassing important data validations, stored procedures, and other means of preserving data integrity. In addition, if you’re connecting commercial systems and any change their database schemas (this often happens when systems are upgraded), your interfaces can suddenly stop working or return erroneous results.
Bottom Line: This approach could be a good fallback if you know exactly what you’re doing.
SCREEN SCRAPING
Often used when connecting mainframe or Web-based systems, screen scrapers simulate actual clients logging into the systems you wish to integrate—for example, screen-scraping “agents” can log on to a mainframe-based HR system as a user, pull out teacher or administrator data, and then pass the information on to the payroll system.
Pro: If you don’t have access to source code or APIs, and you either can’t get to the underlying data, don’t have a schema for it, or just don’t want to implement data-level interfaces, screen scraping can be an effective method. Almost any Web-based or terminal-based application, and even some client-server programs, can be accessed this way.
Con: If any back-end application changes, you’ll have to rescrape all interfaces to that system. And if you want to do any transformations, processing, or operations on the data as it travels between systems, you’ll need to have program code, which can sometimes be in proprietary or obscure scripting languages.
Bottom Line: Sometimes this is your only option—if so, you’ll be glad to have it.
THE EI APPROACH
So if custom interfaces are sometimes problematic, is there another solution? An increasing number of companies, including WebMethods and IONA, are selling dedicated enterprise application solutions designed to connect back-end systems in a standard, robust, and reliable way. Think of EI systems as the difference between a free-for-all classroom discussion and a moderated one—with a central source facilitating all the connections, the total number of interfaces you need is much smaller, and it’s, in theory, a much more efficient way to manage connectivity. But EI products aren’t cheap—you can easily expect to pay prices in the hundreds of thousands of dollars for a large-scale EI system. If you have a small number of systems to connect, you’re better off sticking with one of the custom interface methods described above. But since the complexity of creating and maintaining custom interfaces increases exponentially, at some point you cross the return-oninvestment threshold where you can save significant money (and sanity) by investing in an EI system.
Pro: Most EI solutions come with a huge variety of preprogrammed interfaces to many common back-end systems, as well as comprehensive tool kits used to build new interfaces for less-common systems. And if one of the back-end systems ever changes, all you typically have to do is modify the single interface between that system and the EI system instead of changing each individual custom interface you’ve written.
Con: Aside from the initial cost, EI systems are relatively complex to install and require well-trained staff to write and maintain the interfaces. This is a plausible proposition for large districts but small districts may need to look into outsourcing their EI development. Most EI vendors will happily provide this expertise, but again, it won’t come cheap. Finally, like most complex system deployments, without careful planning, it can fail spectacularly. Bottom Line: If you can afford it, EI can help you increase operational efficiency, decrease maintenance costs, and deliver new services. Moreover, it can let your IT systems cleanly integrate into the kind of data warehouses and other systems that can help you cope with No Child Left Behind requirements.
Richard Hoffman, former Web technologies coordinator for Fairfax County Public Schools in Virginia, is a technical architect based in New Hampshire. He has written numerous articles for Technology & Learning and Network Computing.
