Between 1992 and 2011 we had the pleasure of running a billing support system for a large telecomm company. The contract started off between us and GTE Labs when they were called in to help speed up processing for past due accounts. Over time we switched from working with Labs to working directly with receivables management, the part of the company that handles past due bills.

Originally the process was a simple one to automatically review accounts based on their appearing on an aging report. If the account met certain criteria, we automatically performed a “temporary disconnect” to shut off the customer’s phone until they paid their bill. The problem to this point was that the reports listing the past due accounts were printed and each one manually reviewed by a telephone representative to see if the customer should be “treated” (in this case, that means disconnected until payment).

The problem was that they couldn’t keep up with the volume and so fell further and further behind. Our solution was to set up small Unix servers in the customer contact facilities. These servers had synchronous serial cards that communicated with their mainframes and pretended to be a printer (so we could get the reports of overdue accounts) and as a representative so we could examine accounts online and check to see if there was any reason not to disconnect them.

These two techniques are “print scraping” and “screen scraping” respectively. Both of them count on the fact that classic mainframe programs are very rigid in their input and output. The reports generated followed a pattern so that once you detect headings in the reports, you can then determine where each data value appears on the page.

Similarly the screens used were IBM 3270 style displays, 24 x 80 screens with a mixture of “protected” and “unprotected” fields. Working with these involved typing input or commands into the unprotected fields and then pressing a function key to transmit the changed fields to the mainframe.

Over time our system took over more and more interpretive functions, some as simple as reading reports and summarizing them, some as complicated as cloning existing telephone account records.

As time went by we switched from Unix to Windows, from serial communications to TN 3270 (a standard Telnet protocol for communicating with mainframes) and most importantly to using XML as our definition language for our screen and print scraping techniques. In our final versions the XML would be compiled into C++ pseudo code for use in our libraries. This gave us the ability to quickly adapt to undocumented changes on the mainframe yet keep up the high speed processing we needed.

By mid-2010 all of our processing had switched to automatic service order generation with most of the billing and interpretation done externally. By the end of 2010 all processing had been brought in-house.