Latest Entries »

“What we have here is a failure to communicate.” -Captain

Rackspace played an important role in part of the tech stack I implemented for many of my IT customers for nearly ten years. We started implementing Rackspace’s Hosted Exchange solution back before Microsoft Office 365 hit its stride, and their service offering was truly first-rate at the time.  Unfortunately, that time is gone, punctuated by Friday’s dismal service breakdown and Rackspace’s complete failure to communicate with their customers in real-time as things unfolded.

If I am managing the Exchange server for a single company, never mind thousands of companies – which is likely what Rackspace is doing – and that server is not working, I have one responsibility that is just as important as getting the server back online. I must communicate with managers to give them information about what is going on to create reasonable expectations for when and how the issue will be resolved and facilitate their ability to mitigate risk.  In a normal situation, doing so makes perfect sense.

There is no good reason that wouldn’t be done.  The fact that this wasn’t done throughout the day on 12/2 can only mean a few things: absolute chaos, inadequate staffing, lack of information or perhaps some of each of those things.  Almost anyone managing IT and Exchange knows this.  I realize that Rackspace was likely determining the scope and severity of the issue, but in not communicating anything meaningful for the entire business day, Rackspace failed its customers.  They put the IT workers who support their solution in the unenviable position of only being able to communicate to their managers and customers that Rackspace wasn’t communicating with them.

To those who called Rackspace multiple times, listened to incessant jazzy hold music, and kept a vigilant eye on their status page most of the day, it no doubt became clear that this issue wasn’t something they could count on Rackspace to resolve in the short-term.  We will eventually know more about what happened, but the real story so far is Rackspace’s poor communication about what was going on in the moment.

For those still monitoring the status at status.apps.rackspace.com on 12/3, there was an update at 1:57am.  Any lingering hope of Rackspace resolving the issue sometime soon died with this update: “security incident … do not have an ETA for resolution … may take several days” So too would any other plans that IT workers utilizing Rackspace as part of their tech stack to provide Hosted Exchange had for their weekends.

The full message as provided from Rackspace at 1:57am on 12/3 follows.

What happened?

On Friday, Dec 2, 2022, we became aware of an issue impacting our Hosted Exchange environment. We proactively powered down and disconnected the Hosted Exchange environment while we triaged to understand the extent and the severity of the impact. After further analysis, we have determined that this is a security incident.

The known impact is isolated to a portion of our Hosted Exchange platform. We are taking necessary actions to evaluate and protect our environments.

Has my account been affected?

We are working through the environment with our security teams and partners to determine the full scope and impact. We will keep customers updated as more information becomes available.

Has there been an impact to the Rackspace Email platform?

We have not experienced an impact to our Rackspace Email product line and platform. At this time, Hosted Exchange accounts are impacted, and not Rackspace Email.

When will I be able to access my Hosted Exchange account?

We currently do not have an ETA for resolution. We are actively working with our support teams and anticipate our work may take several days. We will be providing information on this page as it becomes available, with updates at least every 12 hours.

As a result, we are encouraging admins to configure and set up their users accounts on Microsoft 365 so they can begin sending and receiving mail immediately. If you need assistance, please contact our support team. We are available to help you set it up.

Is there an alternative solution?

At no cost to you, we will be providing access to Microsoft Exchange Plan 1 licenses on Microsoft 365 until further notice.

To activate, please use the below link for instructions on how to set up your account and users.

https://docs.rackspace.com/support/how-to/how-to-set-up-O365-via-your-cloud-office-control-panel

Please note that your account administrator will need to manually set up each individual user on your account. Once your users have been set up and all appropriate DNS records are configured, their email access will be reactivated, and they will start receiving emails and can send emails. Please note, that DNS changes take approximately 30 minutes to provision and in rare cases can take up to 24 hours.

IMPORTANT: If you utilize a hybrid Hosted environment (Rackspace Email and Exchange on a single domain) then you will be required to move all mailboxes (Rackspace Email and Exchange) to M365 for mail flow to work properly. To preserve your data, it is critical that you do not delete your original mailboxes when making this change.

I don’t know how to setup Microsoft 365. How can I get help?

Please leverage our support channels by either joining us in chat or by calling +1 (855) 348-9064. (INTL: +44 (0) 203 917 4743).

Can I access my Hosted Exchange inbox from before the service was brought offline?

If you access your Hosted Exchange inbox via a local client application on your laptop or phone (like Outlook or Mail), your local device is likely configured to store your messages. However, while the Hosted Exchange environment is down, you will be unable to connect to the Hosted Exchange service to sync new mail or send mail using Hosted Exchange.

If you regularly access your inbox via Outlook Web Access (OWA), you will not have access to Hosted Exchange via OWA while the platform is offline.

As a result, we are encouraging admins to configure and set up their user’s accounts on Microsoft 365 so they can begin sending and receiving mail immediately. If you need assistance, please contact our support team. We are available to help you set it up.

Will I receive mail in Hosted Exchange sent to me during the time the service has been shut down?

Possibly. We intend to update further as we get more information.

As a result, we are encouraging admins to configure and set up their user’s accounts on Microsoft 365 so they can begin sending and receiving mail immediately. If you need assistance, please contact our support team. We are available to help you set it up.

IT workers likely spent much of Saturday and Sunday migrating email to another provider, such as Microsoft, and some may still not be done today.  Depending on the readiness of contingency plans in place at various firms and/or the extent of local OST caching some firms may now be depending on Rackspace to recover their email records.  It is a little late to look at the SLA, but it is probably worth another glance now.

Though nearly all investment professionals utilize email journaling due to compliance requirements, I am not sure that everyone doing so has a complete backup of their current active email accounts.  They may have the ability to query their email records for compliance analysis using the journal but recovering all of the records that were stored at Rackspace as they were on 12/1 may be more complicated and drawn out.

Based on what customers currently know, it is possible that some users may not be able to recover some emails.  Remember that users are waiting for Rackspace to resolve a security issue.  Security is as much about protecting data from being lost as it is about it being compromised.  So there may be an issue with data loss rather than potential hacking that could have exposed passwords or data.  Rackspace hasn’t divulged the exact nature of the security incident.

One obvious takeaway from this issue is that you should be locally caching all Exchange data for your account in your local environment if you can.  To check your settings in Outlook, you can navigate to the screen shown below in Outlook by doing the following:

  1. Click on File, Account Settings, Account Settings (again).
  2. Select the email account you want to verify and click on the Change button.
  3. The default for downloading email for the past is typically “1 year.” If yours is set to “1 year”, you probably want to drag the control to the right to until it says “All” as shown below; however, I would defer to your IT people on this, because if they aren’t downloading all of your data, they could have a good reason.
  4. Once you have updated the setting, click the next button and then done button to commit the changes.

Migration, Initial Recovery and Complete Recovery

For the companies faced with this issue, restoring complete functionality of email and supporting applications will take time. If they haven’t already, they need to initiate migration by redirecting their DNS records so that email flows to another service provider and perform an initial recovery to get email running on computer/phones. They may also need to do a more complete recovery that includes all of the records that were stored in the users’ email and any specific email profile configuration settings that might have been lost.

Assuming the migration process goes smoothly, my estimation of the time required is roughly 2+ hours to update the DNS records necessary to point your email to a new service provider, wait for that info to propagate, and make sure all users are set up in the new service provider’s environment and everything is working properly.  Let’s be pessimistic and say this takes four hours.  Beyond that, you would still need to do the following items for each individual user:

  1. Have a backup of the PST on hand and ready to import, or create one from existing cached copies.
  2. Create new mail profiles to replace individual accounts within the current email profile. (My recommendation would be new profiles because I would want to maintain the old ones with their email records.)
  3. Depending on how things are configured, that might be a process that you would have to do once per user, or multiple times if they have notebooks and desktops with separate email profiles.
  4. Additionally, any mail accounts on Apple iOS and Android devices would need to be deleted and recreated.

Expecting to spend less than an hour per user on average to do this would be overly optimistic, but two is probably a reasonable guesstimate and some of the processing could likely be accomplished for various users simultaneously. But things like this almost never go smoothly.  These times could potentially be reduced through the use of third-party tools and automation, but let’s assume you don’t have access to those. A relatively small ten-person office that was using Rackspace could require 24 hours of IT work done over the weekend to bring them back online with most of their email on a new service.

What happened with Rackspace should also be a wake-up call to firms utilizing any cloud services and depending on them for real-time business continuity without necessarily having a full understanding what will happen in certain contingency scenarios.  Any service, whether it is cloud-based or on-premise, is only as good as the people managing it and your SLA.

Thankfully, the number of customers I service with a dependency on Rackspace has shrunk to almost none. Most have moved to Office 365.  Given this latest issue, it appears to me that Rackspace has been treading water with their Hosted Exchange service for the past year or so.  During that time using Multi-Factor Authentication (MFA) with email has become a critical business requirement and Rackspace hasn’t answered that call on their Hosted Exchange platform.  Their recommended solution for Hosted Exchange customers has been to buy Office 365 via Rackspace to get that MFA functionality from Microsoft.

To Rackspace’s credit, they did eventually start to give more useful information and constructive advice regarding the situation at 8:19 pm EST on Friday, but they went a whole day without providing anything of note. I don’t think I have ever seen a critical IT issue handled quite this way. If you are dealing with a Rackspace employee today, or with someone at your office who has been impacted by this event, try to be patient and kind. Doing anything else is pointless and counterproductive. These people are in an unpleasant and untenable situation today.


Kevin Shea Impact 2010

About the Author: Kevin Shea is the Founder and Principal Consultant of Quartare; Quartare provides a wide variety of technology solutions to investment advisors nationwide.

For details, please visit Quartare.com, contact Kevin Shea via phone at 617-720-3400 x202 or e-mail at kshea@quartare.com.

With my long-standing history as a seasoned and impartial technology consultant catering to the wide-ranging needs of Advent users, it should come as no surprise that companies that have moved away from Advent call me to assist them if they have Advent specific needs after their agreements with Advent have lapsed.  In those specific cases, I suspect my independence from Advent is one of the most appealing features of my service, but many Advent users that have ongoing agreements with Advent also retain me to provide a level of service that Advent seems unwilling or unable to provide.

One of the things I get regular calls about is getting Axys running again.  These calls occur either when firms upgrade their servers or when firms that have moved on to competing Portfolio Management Systems dust off their old Axys files with hopes of tapping into Axys again.  My experience consulting to financial services firms using Advent Software for thirty-plus years facilitates my ability to resolve issues like these easily. 

Many of those calls I get start with the caller telling me, “We reinstalled Axys on the server and it isn’t working.”  And inevitably, this tells me more about the underlying issue than the caller ever could.  You certainly can reinstall Axys, but you probably don’t need to because Axys on the server is just a bunch of files that you access from another PC.  The most important thing to keep Axys working properly aside from the proper installation being done (at some point in the past) is making sure that users have all necessary rights to the shared folders.

This article is focused on explaining what the requirements are to empower you or your firm to resurrect Axys.  As usual, I’ll be providing a level of information in this piece that may be more than you need to solve any immediate problem with the hope that info is useful to you in the future.

Axys Versions

There are two fundamental versions of Axys: the multi-user version and single-user version.  To add a little confusion, the multi-user version is frequently referred to as the network version, but both fundamental versions are regularly installed on networks.  So, the network version is a bit of a misnomer.  Among these two fundamental versions, there is also the version of the software, which is at this point typically version 3.8, 3.8.5, 3.8.6 or 3.8.7.  In addition to these, there are also Monocurrency, Multicurrency and Variable Rate versions, to name a few.  Suffice to say, there are a lot of different versions.

Axys Licensing Model

The concurrent licensing model that Axys implements applies to both single-user and multi-user versions.  In both instances, the number of real Axys users typically exceeds the total licensed users, but having a multi-user version allows more than one user to use Axys simultaneously and adds certain multi-user features, such as user-specific settings and separate blotters, et cetera.

Understanding How Axys is Installed

Initially, the single-user version is simpler to install because the primary program (Axys) and supporting programs (Dataport, Data Exchange, Report Writer, et al.) hypothetically only need to be installed once.  That would be true if there literally was only one user using the software on one PC.  In actuality, the single-user version of Axys and supporting programs get installed multiple times in a network environment. They need to be installed once for every user, albeit to the same destination for each user (e.g., F:\Axys3).

During the Axys install process, certain required files are copied to the user’s PC and/or profile and Axys creates registry keys in HKEY_CURRENT_USER\SOFTWARE\Advent.  The most critical Axys registry keys are stored in HKEY_CURRENT_USER\SOFTWARE\Advent\Axys\3.  Although there are several important Axys files, the firmwide.inf is perhaps the most crucial file.  In a single-user installation, this text file, which can be found in the root folder of Axys (e.g., F:\Axys3), details certain settings in use and where all of the other Axys files can be found.

The multi-user version must also be installed multiple times for users, but the initial Axys install varies.  You install it once to the network/primary destination folder (e.g., F:\Axys3) and then install it again for the rest of the users (e.g., F:\Axys3\users\kevin where a firmwide.inf file will be created).  Similar to the single-user version, the supporting programs such as Dataport, Data Exchange and Report Writer would also need to be installed if the user needs those, or if you are trying to make sure all of the users have access to all of the supporting apps. The same registry keys are used for the multi-user install as the single-user version, but the multi-user (a.k.a. network) version adds an additional critical file: the netwide.inf file.

Netwide.inf versus Firmwide.inf

These two files are closely related.  The netwide.inf file should only be found in the root Axys folder of a network install, but firmwide.inf files exist in both single-user and multi-user environments.  The multi-user version is designed to use the settings in the netwide.inf as the system default and have any settings in the firmwide.inf supersede the settings in the netwide.inf.  As a rule, you should never see a firmwide.inf in the root Axys folder of a network install.  You should also almost never see a netwide.inf file in the root of a single-user Axys installation.


A Recurring Axys Installation Bug

With regard to installing Axys, there is a rather annoying issue that has been going on for several years.  It seems that the Axys install will not recognize certain network locations and/or mapped drives.  The fix requires the following registry settings:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System]

“EnableLUA”=dword:00000001

“EnableLinkedConnections”=dword:00000001

Once those settings have been applied, the Axys install program will be able to find the mapped drives.  It seems to me that this is an issue Advent should have addressed a long, long time ago.

Understanding Those Axys Shortcuts and Corresponding Registry Entries

The working folder of the Axys shortcut needs to point to the appropriate folder for the firmwide.inf file.  That means that an Axys shortcut for a single-user version of Axys should have a “Start in” folder like F:\Axys3, whereas the multi-user version would have “Start in” folder like F:\Axys3\users\kevin.  Assuming the same install folder was used, the target for these shortcuts would be the same: F:\Axys3\Axys32.exe.  Likewise, the registry entries associated with Axys should match these settings.  When I am looking at a system, I can usually determine if Axys has been installed properly by looking for consistency between the shortcuts and the following registry entries: ExePath, NetPath and UserPath.

In summary, your Axys install is dependent on a few things: the files themselves, access to the location where they are stored and proper mapping to the location of those files in the registry, firmwide.inf and netwide.inf if applicable.  Hopefully, you can get things back online on your own, but if you need assistance with your Advent installation, reach out to me and I’ll do my best to assist you.


Kevin Shea Impact 2010

About the Author: Kevin Shea is the Founder and Principal Consultant of Quartare; Quartare provides a wide variety of technology solutions to investment advisors nationwide.

For details, please visit Quartare.com, contact Kevin Shea via phone at 617-720-3400 x202 or e-mail at kshea@quartare.com.

Schwab is no longer providing price file downloads that some Axys users have relied on for decades.

When I set out to write this, I had some trouble deciding on the title.  At first, I considered “Schwab Hamstrings Pricing for Advent Users”, but that’s inflammatory and not entirely accurate, so I couldn’t do that.

I could just as easily have titled this blog “Stubborn Axys Users Refuse to Embrace Benefits of ACD Interface” or “Axys Users Slow to Hire Consultants to Address Schwab Point-to-Point Interface Changes”, but in truth Schwab is discontinuing their support for Axys in the data they provide directly to their customers via the Schwab download, so I had to go with “Schwab Discontinuing Support of Axys Point-to-Point Interface – Again?”  Besides, picking one of those other titles would have made me write a blog with a different message.  In advance, I’d like to clarify that this issue only impacts the Schwab point-to-point interface and has no effect on those that receive their Charles Schwab data from Advent’s ACD interface. 

I need to apologize to those who have read my blog regularly in the past.  First, I am sorry I haven’t posted anything in a while.  Additionally, I must apologize that this blog may not seem particularly newsworthy for some.  You may even be thinking, Didn’t this happen eons ago.  The answer is yes and no.

About twenty years ago, there was some drama about the point-to-point interface that Charles Schwab provided to its customers and Advent, being Advent, may have been perceived as attempting to screw Charles Schwab and its customers to make more money.  Schwab, being Schwab, sued Advent – to paraphrase the judge told Advent, “You can’t do that.”

According to what I can dig up now, the firms quit wasting each other’s time and money nearly seven months after that preliminary injunction, coming to a compromise that allowed Schwab to continue to provide their point-to-point data for a period of time.  In my recollection of the events, it seemed much more drawn out.  Fast forward twenty years, and now everyone that is still relying on this particular set of data directly from Schwab’s download is back to square one.

Back in 2002, the underlying issue was that Advent didn’t want Schwab to continue providing the data without going through ACD and Schwab wanted to continue providing the data to satisfy their existing customers, who had grown dependent on getting that data via the point-to-point interface.  From the perspective of those Axys customers, it is easy to understand their position then and now…  it is pretty much free, and it works.  Why would we want to change that?

Somehow, for more years than I would have thought possible these holdouts that either saw no reason to fix something that wasn’t broken or were too cheap to move to ACD continued to do what they had been doing for decades.  I never thought this would have gone on as long as it did.  Alas, as they say, all good things must come to an end.  That is apparently what is happening now.

Schwab is in the process of stopping production of the files that feed Dataport for this subset of Axys users.  Last month, they stopped producing the price (CSMMDDYY.pri) files; they are also planning to stop producing other key files, such as transactions, sometime in 2023.  The sudden inability to create a price file no doubt caused some difficulties for those still dependent on them.  As a result, a couple firms reached out to me.

After a brief discussion with the first firm, I agreed to automate the creation of the missing price file.  According to my customers, both Advent and Schwab were unwilling to assist them with the issue.  Advent’s not planning to make changes to their interface to take in the new Schwab files, and Schwab’s not planning to help clients transform the files into something that can be ingested directly into Dataport.

It sounded way too easy for someone with my experience, and I thought it would only take a “few” minutes.  Somewhat embarrassingly, I spent a few hours creating the automation necessary to do the translation.  However, in a subsequent implementation for another customer, I was able to have a meet-and-greet call with them and a follow-up call to implement and test the solution in their environment very quickly.  All of it was accomplished in a couple hours, and on the very same day the prospective customer contacted me, leading me to believe that future implementation may be performed in a matter of minutes.

Those dealing with what is currently limited to a pricing issue have a handful of choices, none of which are fun to deal with when you need yesterday’s prices now:

  1. The most obvious choice: consider implementing the Schwab Interface via ACD.  It might be worth it.  I am not kidding.  I have plenty of clients that use ACD.
  2. Use a third-party pricing service like IDC/ICE or Telemet.
  3. Key the prices in manually.  I am not recommending this, but it is certainly an option.
  4. Utilize automation to recreate the missing price file (CSMMDDYY.pri) from the security file (CRSYYYYMMDD.SEC) now provided by Schwab.  This isn’t very difficult, and that is what I have done for those who have asked me to resolve the issue for them.

Addressing the pricing issue alone is a stop-gap solution at best.  The larger issue down the road is translating the transaction files, which will need to be done in 2023.  At my clients’ request, I have agreed to look into doing this for them as well, and I will most likely do it.  With my experience building Axys interfaces and doing the requisite transaction mapping et cetera it probably won’t be that big a deal, but it will certainly be more complicated and time-consuming than the Price File Translator was to create.

As always, if this issue is something your firm needs assistance with, please feel free to contact me directly.


Kevin Shea Impact 2010

About the Author: Kevin Shea is the Founder and Principal Consultant of Quartare; Quartare provides a wide variety of technology solutions to investment advisors nationwide.

For details, please visit Quartare.com, contact Kevin Shea via phone at 617-720-3400 x202 or e-mail at kshea@quartare.com.

IMG_8217

I like to think that the greater purpose of much of the work I do with investment professionals is to simply show them what is possible. If you understand what can be done and how, I trust that you will have no problem coming up with many things that you want to do. Process automation, workflow improvements, integration, custom apps, and enhanced reporting are all easier to imagine and implement in your own environment when you have the benefit of the insights and perspective of relevant and significant user experience.

Like many before it, this article was written with that idea in mind. There is a very practical component (the source code below) which a small number of users may be able to take advantage of; the larger audience of managers, investment operations and technology professionals will hopefully come away with the idea of what can be done and roughly how it can be done, without necessarily planning to do this particular thing. More likely, readers will file away the knowledge of this article and revisit it later.

Most readers of this blog will not read it on the day I post it, or any time soon. Some day in the future, when the information in this particular blog becomes relevant to them, they will run a search and eventually find this article.

Audit Trail Usage

The Audit Trail (didpost.aud) file, a critical component of Advent APX and Axys, empowers users to review any transactions posted to portfolios in the system.  It can also facilitate small- and large-scale removal of transactions that are posted in error. Among some Advent users, proactive management of the audit file is almost nonexistent until they hit the wall by letting the file grow to an unmanageable size or find it useful and perhaps even necessary to create subsets of the file as part of an audit.

One approach to proactively managing the file in Axys is to have a routine process whereby the file is copied quarterly or monthly and the current file is deleted.  In this scenario, new transactions posted create a new Audit Trail file.  And if you need to reference older transactions, you can temporarily copy the current file to another file and restore the file you need to access as the didpost.aud.  When you are done accessing the old data, you revert things back by copying the original Audit Trail back to didpost.aud. Though the process is a bit clunky, somewhat risky, and manual, there are probably many firms using this approach.  You could easily automate the processes I have described to create a more elegant solution.

In the past, I created some automation to routinely export the audit trail to a CSV file and create separate audit files of those exported CSVs for pre-selected periods.  The code to do this is not complex, so I doubt I am the only one who has implemented this approach.  This worked fine for many years, but it stopped working at some point.  I am not sure when it happened, but I found that the ability to export the audit trail file in Axys 3.x via IMEX no longer worked reliably.  Any attempt to export the Audit Trail would abend, leaving you with only part of the exported file you needed.  Initially, I wasn’t sure whether the issue was limited to one of my customers, but after reviewing a few sites I work with, I found that the ability to export large audit trail files using IMEX no longer worked across the board. The issue is an Axys issue and not necessarily an APX issue, but I haven’t tried to export the audit trail in APX recently.

This failure to export Audit Trails posed quite a problem.  Though the Audit Trail could be reviewed, and copy functions within the review windows worked, the size of the files made any kind of manual selection process onerous and prone to error.  Besides, what would you do if you did select the data?  I guess you could copy it into an empty trade blotter and save the blotter to a different file, but the trade blotter isn’t an Audit Trail file.

My assumption was that the integrity of the Audit Trail file was intact, but the function to export a larger file was broken.  Usually, I frown upon working with Advent files directly.  I prefer to work with exported files and consider that a best practice to avoid issues that can occur when the format of Advent files ultimately changes in later versions.  In this particular instance, I was forced to read and write AUD files directly since the export mechanism wasn’t working properly, but I was happy with the end result and thought the source code might be useful to others.

Source Code to Fix Axys Audit Trail Export Issues

The code below reads the Audit Trail (didpost.aud) file in its native format, creates a time-stamped backup of the original file, and creates an annual didpost file for each year in a backup folder within the f:\axys3\aud folder.  As some of you consider using this code, you may be concerned. After all, the integrity of the Audit Trail is not really something you want to mess around with lightly. You may even be thinking, Just because you can doesn’t mean you should. Nevertheless, I created, carefully tested, and successfully implemented the code to fix the audit trail for some Axys users that realized they had this issue.

Imports System.IO

Module Module1

  Sub Main()

    ' This subroutine was created to address a problem in Axys 3.x.
     
    ' The routine serves two purposes. First of all, it addresses
    ' a problem where the Audit Trail can no longer be exported in
    ' its entirety.  The secondary purpose of the routine is building
    ' annual Audit Trail files for record-keeping purposes.
   
    ' This process is intended to be run annually.  When run it creates
    ' a backup of the existing Audit Trail and then create separate
    ' Audit Trail files for each of the previous years.  However,
    ' given critical nature of the Audit Trail I recommend that you
    ' make sure you have additional backups of the Audit Trail to
    ' ensure that you can return the to original if necessary. When
    ' the process is completed the current Audit Trail will only have
    ' the records that were posted in the current year.  Please note
    ' this code should be used carefully by users that completely
    ' understand its purpose.

    ' written in Microsoft Visual Studio's Visual Basic by
    ' Kevin Shea (aka AdventGuru) & updated 06/27/2019

    ' Disclaimer: This routine works fine for the specific instance
    ' it was created for, but could need additional modifications
    ' for different circumstances.

    Dim idate As String
    Dim d As Date
    Dim i As Long = 0
    Dim c As Integer
    Dim x As Integer
    Dim bufferpointer As Long = 0
    Dim ByteCount As Long = 0
    Dim CurrentYear As String = "0000"
    Dim NewYear As String = "1111"
    Dim ByteStop As Long = 0
    Dim ByteStart As Long = 0
    Dim size As Long
    Dim posts As Integer
    Dim InitialPostDate As Date
    Dim semicolon As String = ";"
    Dim CommentHeader() As Integer = {254, 250, 251, 252, 89, 0, 48, 0, 59, 0} '10 bytes
    Dim FileHeader() As Integer = {97, 117, 100, 49, 46, 48, 49, 48} '8 bytes
    Dim AuditTrailFolder As String
    Dim AuditTrailBackupFolder As String
    Dim AuditTrailFile As String
    Dim time As DateTime = Date.Now
    Dim format As String = "M_d_yyyy_HHmm"

    AuditTrailFolder = "f:\axys3\aud"
    AuditTrailBackupFolder = "f:\axys3\aud\backup"
    AuditTrailFile = "didpost.aud"

    System.IO.Directory.CreateDirectory(AuditTrailBackupFolder)
    MsgBox("Please make sure no users are posting blotters before continuing ...")

    My.Computer.FileSystem.CopyFile(AuditTrailFolder + AuditTrailFile,
    AuditTrailBackupFolder + time.ToString(format) + ".aud")

    MsgBox("Audit Trail Backup complete. Click okay to continue and analyze file.")

    Dim sample As String
    sample = Space(11)
    Dim bytes() As Byte
    bytes = File.ReadAllBytes(AuditTrailFolder + AuditTrailFile)
    size = My.Computer.FileSystem.GetFileInfo(AuditTrailFolder + AuditTrailFile).Length

    MsgBox("File is " + size.ToString + " bytes long.")

    'put the file header in the buffer
    Dim buffer(size) As Byte

    For i = 0 To 7

      buffer(i) = FileHeader(i)

    Next i

    bufferpointer = 8
    ByteCount = 0
    ByteStart = 8

    Maxloop:

    Do While ByteCount < size

      ByteCount = ByteCount + 1
      c = 0

      For x = 0 To 9

        If ByteCount + x = size Then Exit Do
        If bytes(ByteCount + x) = CommentHeader(x) Then c = c + 1

      Next x

      If c = 10 Then

        If Strings.Chr(bytes(ByteCount + 84)) = semicolon And Strings.Chr(bytes(ByteCount + 32)) = "a" And Strings.Chr(bytes(ByteCount + 33)) = "t" Then

          idate = (Strings.Chr(bytes(ByteCount + 23)) + Strings.Chr(bytes(ByteCount + 24)) + "/" + Strings.Chr(bytes(ByteCount + 26)) + Strings.Chr(bytes(ByteCount + 27)) + "/" + Strings.Chr(bytes(ByteCount + 29)) + Strings.Chr(bytes(ByteCount + 30)))

        End If

        d = Convert.ToDateTime(idate)
        posts = posts + 1
 
        'if post date is greater than threshold of collection save buffer as distinct audit trail and reset buffer file
        NewYear = d.ToString("yyyy")

        If posts > 1 And (NewYear <> CurrentYear) Then

          'a change in the current year to a new year triggers audit trail creation for the current year
          ByteStop = ByteCount - 1

          For i = ByteStart To ByteStop

            buffer(bufferpointer) = bytes(i)
            bufferpointer = bufferpointer + 1

          Next

          Dim s As New System.IO.FileStream(AuditTrailFolder + CurrentYear + ".aud", System.IO.FileMode.Append, System.IO.FileAccess.Write, System.IO.FileShare.ReadWrite)

          s.Write(buffer, 0, bufferpointer)
          s.Close()

          ByteStart = ByteCount
          CurrentYear = NewYear
          bufferpointer = 8

        End If

      End If

      If posts = 1 Then

        InitialPostDate = d
        CurrentYear = InitialPostDate.ToString("yyyy")

      End If

    End If

  Loop

  'We did everything, but the current year.  Now let's do that too.
  For i = ByteStart To size - 1

    buffer(bufferpointer) = bytes(i)
    bufferpointer = bufferpointer + 1

  Next

  buffer(bufferpointer) = 0

  Dim f As New System.IO.FileStream(AuditTrailFolder + CurrentYear + ".aud", System.IO.FileMode.Append, System.IO.FileAccess.Write, System.IO.FileShare.ReadWrite)

  f.Write(buffer, 0, bufferpointer)

  f.Close()

  MsgBox(posts.ToString + " posts analyzed.  The first post was " + InitialPostDate.ToString("MM/dd/yyyy") + ". The last post was " + d.ToString("MM/dd/yyyy") + ".")

  MsgBox("File copy with binary read all bytes and write all bytes complete.")

  End Sub

End Module

Everyone has a different level of interest in the tech end of things. Some people want to understand it completely; others don’t want to know anything about it. I suspect that most of my readers fit somewhere in between those two extremes. On any given day, depending on what else is on your plate, you may lean towards one end or the other. In some cases, it simply makes more sense just to have someone like me take care of these things. If you have a problem with your Audit Trail that needs to be resolved, it may be one of those things.


About the Author: Kevin Shea is the Founder and Principal Kevin Shea Impact 2010Consultant of Quartare; Quartare provides a wide variety of technology solutions to investment advisors nationwide.

For details, please visit Quartare.com, contact Kevin Shea via phone at 617-720-3400 x202 or e-mail at kshea@quartare.com.