A story about following the problem

I Just Wanted to Make Things Work Better.

How a frustrating reporting process led me from automating browser clicks to building desktop applications.

Read the story Personal essay · An evolving visual story

I didn't set out to become a programmer. I was just tired of wasting time doing things that seemed like they could be done a better way.

When I was working as an EVMS analyst at Austal USA, part of my job involved preparing weekly reports for the projects I supported. The process wasn't particularly difficult, but it was tedious.

I'd log into Empower, open the appropriate view, and export the data to Excel. Then I'd open the exported file, remove unnecessary rows, apply filters, and clean everything up before importing it into Microsoft Access. Once the data was in Access, I'd run a series of macros that nobody seemed to fully understand. Then I'd open a completely separate Excel workbook, refresh it twice, and save the finished report.

And I'd have to do that for every project, every week.

What bothered me most wasn't necessarily the work itself. It was how much time I spent waiting. Waiting for exports, waiting for Access, waiting for Excel to refresh. Every step depended on the one before it, and a lot of my time was spent watching things happen rather than actually doing anything.

There was another frustrating part of the process, too. Sometimes the data wasn't even ready in Empower. Unfortunately, the only way to find out was to attempt an export. If it wasn't available yet, I'd have to come back later and try again.

I started questioning why the process needed so many steps. I knew the Access portion shouldn't be necessary, and the final Excel report was based on a template. It seemed reasonable that the data could go directly from Empower into that template without everything in between.

I just didn't know how to make that happen.

A little experimentation

My first attempts were pretty rudimentary. I started playing around with JavaScript bookmarklets to automate some of the things I was doing in Empower. I wanted the browser to handle the repetitive clicking, navigation, and exporting for me.

To do that, I needed to understand the elements I was interacting with. I started using Chrome's Inspect Element tool to see what was happening behind the buttons I was clicking.

One day, while exploring Chrome's developer tools, I stumbled across the Network tab. I don't remember exactly what led me there. I think I was just curious and clicking around.

But what I found was interesting.

I could see the requests Empower was making as I interacted with it. I already had a basic understanding of API calls, and the information in those requests looked familiar. There were URLs, parameters, and responses containing data.

That got me thinking.

Instead of automating all the steps I was taking in the browser, what if I could just make those requests myself?

I was already familiar with VBA, so that's where I started.

I spent a lot of time on Stack Overflow and various VBA forums trying to understand how to make HTTP requests from Excel. Eventually, I found the appropriate reference library and started experimenting with reproducing the requests I'd discovered in Chrome.

It wasn't easy.

Authentication was particularly frustrating. It took me a while to figure out how to log into Empower programmatically and maintain an authenticated session. Even after I got it working, there were times when the session would unexpectedly disappear, and I couldn't reliably figure out why.

But eventually, I had something functional.

I built an Excel workbook with buttons and macros that could communicate directly with Empower and retrieve the data we needed. It eliminated much of the manual process, and the EVMS team started using it.

That was my first real contribution to improving the workflow.

Unfortunately, it didn't stick.

The workbook worked, but it was clunky. It was essentially a collection of macros and buttons sitting on an Excel spreadsheet. It wasn't particularly appealing to use, and unpredictable external factors could cause things to break.

I noticed that the analysts gradually stopped using it.

I'd figured out how to automate the process, but I hadn't built something people really wanted to work with.

Learning Python because I needed to

Around this time, I started looking into Python.

One of the things that attracted me was the debugging experience. With VBA, I often felt like I was guessing when something went wrong. I wanted better visibility into what my code was doing, particularly when communicating with Empower.

Python made that much easier.

I started experimenting with the same HTTP requests using the httpx library. The underlying logic wasn't really any different from what I'd already worked out in VBA, but it was much easier for me to investigate problems and understand what was happening.

Of course, retrieving the data was only part of the problem.

The team was expecting an application they could actually use. I couldn't spend months learning Python in isolation and then start thinking about how to build an interface.

So I learned everything at once.

I was figuring out Python, working through the reporting logic, and teaching myself how to build a desktop interface with CustomTkinter at the same time.

I'd get one thing working, discover another problem, and start researching again. Sometimes I'd spend hours trying to figure out something that seemed like it should be simple.

Having previously worked as an EVMS analyst helped tremendously. I already understood the reporting process because I'd done it myself. I knew what the finished reports needed to look like, and I knew which parts of the process were unnecessarily complicated.

The analysts gave me feedback as I developed the application, but I wasn't starting from an unfamiliar business problem.

Over approximately six months, the application gradually came together.

I called it Clarity.

One of the features I was particularly interested in was the ability to see which contracts actually had data available for a given reporting date.

While investigating Empower's network requests, I'd discovered that I could retrieve the available reporting periods for each contract. That meant I didn't have to attempt an export to find out whether the data was ready.

Clarity could check that information in advance.

When an analyst selected a reporting date, the application would show the contracts available for that date. They could select one or several projects, click Run, and have the completed Excel reports generated automatically in a directory of their choosing.

Clarity also remembered their login between uses, so they didn't have to start from scratch every time.

That's where the name came from. I wanted the analysts to have clarity about which reports were ready instead of repeatedly trying exports and hoping the data was available.

When I first released the Python version to the team, it could handle the entire reporting process.

And this time, they kept using it.

The analysts told me how much they appreciated the application, but what mattered more was seeing them actually abandon the old method. Something I'd started experimenting with because I was tired of waiting had become part of how the team worked.

Building something is one thing. Maintaining it is another.

Getting Clarity into the hands of the analysts introduced a completely different set of challenges.

One of the most frustrating was getting asynchronous operations to work properly with CustomTkinter.

Sometimes the interface would freeze while reports were being processed. Occasionally, the entire application would crash.

And now these weren't just problems I encountered while experimenting on my own computer. They were problems affecting people who were depending on something I'd built.

I had to make changes, test them, and distribute updated versions of the application.

This was when I started getting more serious about version control. I learned Fossil and began using it to manage changes to my code.

I also had to figure out how to update Clarity on everyone else's computers. Eventually, I wrote a VBScript updater to handle distributing new versions of the Python application.

It worked, but maintaining all the different pieces was becoming a project of its own.

Somewhere along the way, I developed a strong interest in the idea of a single executable. I wanted to be able to build an application, put one file somewhere, and run it without worrying about everything else that came with it.

That interest would eventually lead me to Rust, although not directly through Clarity.

Another problem, another rabbit hole

By this point, I'd started applying the same approach to other problems at Austal.

One of those projects was EzMRNI, an application intended to improve the process of reconciling materials received but not yet invoiced in IFS.

This project took me into a completely different technical environment.

I started by learning the existing workflow, then used IFS's debugging tools to investigate what happened behind the scenes when the purchase receipt reports were ordered.

That eventually led me to the official IFS development documentation.

I began experimenting with Python and pythonnet to interact with the .NET DLLs that shipped with IFS Apps 10.

I was trying to understand how to use functionality that was already there, rather than building everything from scratch.

As I worked through the documentation and experimented with those libraries, I started picking up C# without really intending to. I was learning enough about the language and the .NET environment to understand how the existing components worked.

Eventually, I realized I could use what I'd learned to build my own DLL.

That's how the C# engine for EzMRNI came about. What started as an attempt to communicate with existing IFS components had gradually turned into developing my own .NET component to handle the operations I needed.

It was another example of how one problem kept leading me somewhere I hadn't expected to go. I hadn't planned on learning C#, much less building a .NET engine. I was just following the problem and learning whatever I needed along the way.

But once I had that working, I ran into a familiar question: how was I going to put this in front of users?

I was still interested in the idea of distributing an application as a single executable, and that led me to Rust.

I'd already developed a fascination with Python's type system. I liked being able to describe exactly what kind of information a function expected and what it would return. There was something satisfying about catching mistakes before the code ever ran.

When I started exploring Rust, that emphasis on types felt familiar.

Then I discovered Slint, a framework for building user interfaces in Rust, and I loved working with it.

I used Rust and Slint to build the frontend for EzMRNI, connecting it to the .NET engine I'd already developed.

It was a challenging project, and I had to learn quite a bit about how different technologies communicate with one another. But it also introduced me to a way of building applications that I genuinely enjoyed.

And it gave me an idea.

Coming back to Clarity

Clarity was still running in Python, and I was still dealing with the complexities of maintaining and distributing it.

After my experience building EzMRNI with Rust and Slint, I started wondering whether I could give Clarity the same treatment.

So I rebuilt it.

What I found particularly satisfying was that I could reproduce the same interface and user experience in Slint. From the analysts' perspective, nothing really changed.

Most of them didn't even realize I'd switched the application from Python to Rust.

But the difference for me was substantial.

The Rust version could be run directly from a network share. When I needed to distribute an update, I could simply replace the executable with the new version.

I no longer needed the VBScript updater I'd written for Python.

Cargo also gave me a more consistent way to build and test the application, which made working on it feel much more manageable.

Interestingly, the biggest benefit of the rewrite wasn't performance. It was the freedom it gave me to keep improving Clarity without worrying so much about how I was going to maintain everything.

I started adding functionality I'd previously wanted to build, including the ability to edit the Excel report templates directly.

The application continued evolving, but the experience for the people using it remained familiar.

Looking back

The interesting thing about this entire experience is that none of it was really planned.

I didn't start out wanting to learn JavaScript, VBA HTTP libraries, Python, CustomTkinter, version control, .NET interoperability, or Rust.

I wanted to stop wasting time on a weekly report.

Each time I solved one problem, I seemed to uncover another. Sometimes it was a limitation of the technology I was using. Sometimes it was something I didn't understand yet. Other times, it was realizing that just because something worked didn't necessarily mean it was a good solution for the people using it.

I think that's what I've enjoyed most about this journey.

Not necessarily the programming itself, but the process of understanding how something works well enough to question whether it could work differently.

And then figuring out how to make that happen.