Logic App Flat File Schemas and BizTalk Flat File Schemas

I recently started working on a new Logic App for a customer using the Enterprise Integration Pack for Visual Studio 2015, I was greeted with a familiar sight, the schema generation tools from BizTalk, but with a new lick of paint Open-mouthed smile

The Logic App requires the use of Flat File schemas, so I knocked up a schema from the  instance I’d been provided and went to validate it against the instance used to generate it (since it should validate).

My Flat File was a bit of a pain to be frank in that it had ragged endings, that is to say some sample rows might look a bit like:

1,2,3,4,5,6

1,2,3,4,5

1,2,3,4,5,6

Which I’ve worked with before….but couldn’t quite remember how I solved exactly other than tinkering with the element properties.

I generated the schema against the lines with the additional column and erroneously set the last field property Nillable to True. When I went to validate the instance lo’ and behold it wasn’t a valid instance and I had little information about why.

So I fired up my BizTalk 2013 R2 virtual machine (I could have used my 2016 one to be fair if I hadn’t sent it to the farm with old yellow last week) and rinsed and repeated the Flat File Schema Wizard.

So I got a bit more information this time, namely that the sample I’d been provided was missing a CR/LF on the final line and that the Nillable I’d set on the last column was throwing a wobbler by messing up the following lines.

Setting the field’s  Nillable property back to false, but it’s Min and Max occurs to 0 and 1 respectively and I had a valid working schema.

So I copied the schema back to my Logic Apps VM and attempted to revalidate my file (with its final line CR/LF amended). To my annoyance, invalid instance!

I was boggled at this quite frankly but some poking around the internet led me to this fellow’s Blog.

https://blogs.msdn.microsoft.com/david_burgs_blog/2018/03/26/generate-and-validate-flat-file-native-instances-from-flat-file-schemas/

In short there’s an attribute added on a Flat File schema which denotes the extension class to be used by the schema editor, when built by a BizTalk Flat File Schema Wizard it’s set to

Microsoft.BizTalk.FlatFileExtension.FlatFileExtension

When generated by the Enterprise Integration Pack it’s

Microsoft.Azure.Integration.DesignTools.FlatFileExtension.FlatFileExtension

Changing this attribute value in my BizTalk generated Flat File schema and presto, the schema could validate the instance it was generated from.

So in short I’ll say the flavour of the Schema designer tools in the Enterprise Integration Pack seem to throw out errors and verbosity a little different to its BizTalk ancestor, it still throws out mostly the same information, but in different places:

  • EIP

You only get a generic error message in the output log. Go and examine the errors log for more information.

image

  • BizTalk

You get the errors in the output log, and in the error log.

image

In my case the combination of my two errors (the flat file being malformed slightly, and my Nillable field change) in the EIP only gave me an error “Root element is missing” which wasn’t particularly helpful and the BizTalk tooling did give me a better fault diagnosis.

On the bright side the two are more or less interchangeable. Something to bear in mind if you’re struggling with a Flat Schema and have a BizTalk development environment on hand.

Enumerating BizTalk 2016 Features for a Command-Line Installation

As with previous versions of BizTalk Server, you can perform the installation using the GUI or a command-line. To use the command-line installation, you’ll need the list of features that can be installed to add to the /AddLocal command-line. The available documentation for a silent installation of BizTalk Server at https://msdn.microsoft.com/en-us/library/jj248690.aspx relate to BizTalk Server 2013 and 2013 R2 (see https://msdn.microsoft.com/en-us/library/mt743078.aspx for ‘BizTalk Server 2016: What’s new, and installation’, then follow the link Appendix A: Silent installation near the bottom of the navigation menu at the left to get to the above page); there’s nothing that I’ve found so far that provides the setup.exe command line switches, or a list of features for use in a silent installation specifically for BizTalk Server 2016. Note that blindly following the previous guidance and using certain specific /AddLocal features results in an installation failure!

Getting hold of the command-line parameters for setup.exe is, of course, simple. Just run setup.exe with the ‘/?’ switch from a command prompt to get the following:

Command Description
/help or /? or /h Help and quick reference option.
/s <Configuration XML file> Silent Installation of features found in Configuration file.
/passive Passive Installation. Only progress bar will be displayed.
/norestart Supress restart.
/forcerestart Always restart after installation.
/promptrestart Prompts before restarting. This option cannot be used with the /quiet option.
/x or /uninstall Uninstalls the product.
/L <Logfile> Writes logging information into a logfile at the specified path. Always uses verbose MSI logging and appends to existing file.
/IGNOREDEPENDENCIES Bypass checks for downloadable prerequisites.
/INSTALLDIR <Install path> Specify the full path to product install location.
/COMPANYNAME <companyname> Sets the company name.
/USERNAME <User name> Sets the user name.
/ADDLOCAL ALL Install all features.
/REMOVE ALL Remove all features.
/REPAIR ALL Repair installation.
/CABPATH <cabfile> Specify a local path to a redistributable CAB file.
/CEIP Opt in to BizTalk Server Customer Experience Improvement Program.

These commands correspond to those listed on the silent installation page for BizTalk 2013 mentioned above with the exception that the final two commands listed on the web page appear to be missing from the above list generated by BizTalk Server 2016.

The /AddLocal command-line parameter details the features that will be installed. On the silent installation web page, there is a link to follow to the list of features (at http://go.microsoft.com/fwlink/p/?LinkID=189319), however if you browse to that page, you’ll notice that it is marked as the features for BizTalk Server 2010. There are issues using some of the parameters for the installation of BizTalk Server 2016, so it seemed worthwhile attempting to enumerate the parameters that are available to a BizTalk Server 2016 installation.

The installation MSI for BizTalk Server 2016 can be opened using Orca (Orca.exe is a database table editor for creating and editing Windows Installer packages and merge modules – see https://msdn.microsoft.com/en-us/library/windows/desktop/aa370557(v=vs.85).aspx for acquisition and installation instructions) to get the list of features. The screen shot below shows a partial view of the ‘Feature’ table from the ‘Microsoft BizTalk Server64.msi’ file:

Orca Features Table for BizTalk Server 2016 MSI

The information in the ‘Feature’ table, along with information gleaned by running the installer, ticking specific single components and then examining the setup log file can be reorganised to give the following:

Feature AddLocal Command
Portal Components BizTalk, WMI, InfoWorkerApps
      Business Activity Monitoring BAMPortal
Developer Tools and SDK BizTalk, WMI, AdapterImportWizard, BizTalkExplorer, BizTalkExtensions, DeploymentWizard, Designer, Development, Migration, MsEDIMigration, MsEDISchemaExtension, MsEDISDK, OrchestrationDesigner, PipelineDesigner, SDK, TrackingProfileEditor, VSTools, WCFDevTools, XMLTools
Documentation Documentation
Server Runtime BizTalk, WMI, Engine, MOT, MSMQ, Runtime
      BizTalk EDI/AS2 Runtime MsEDIAS2, MsEDIAS2StatusReporting
      Windows Communication Foundation Adapter WCFAdapter
Administration Tools and Monitoring BizTalk, WMI, AdminAndMonitoring, AdminTools, BAMTools, BizTalkAdminSnapIn, HealthActivityClient, MonitoringAndTracking, PAM
      Windows Communication Foundation Administration Tools WcfAdapterAdminTools
Additional Software BizTalk, WMI, AdditionalApps
      Enterprise Single Sign-On Administration Module SSOAdmin
      Enterprise Single Sign-On Master Secret Server SSOServer
      Business Rules Components RulesEngine
      MQSeries Agent MQSeriesAgent
      BAM Alert Provider OLAPNS
      BAM CLient FBAMCLIENT
      BAM-Eventing BAMEVENTAPI
      Project Build Component ProjectBuildComponent

Notes:

  • The ‘BizTalk’ and ‘WMI’ feature are specified in numerous places in the table above. You only need to specify each of these items once.
  • The parameters are case sensitive. Specifying a parameter incorrectly, e.g. OlapNs rather than OLAPNS will result in a silent installation failure.
  • When adding the parameters to the command line, it is important that there is no space between the items. Including a space (e.g. ‘BizTalk, WMI, AdditionalApps’ rather than ‘BizTalk,WMI,AdditionalApps’) will result in a silent installation failure.
  • One of the features, ‘SDKScenarios’, is never mentioned in the setup log file. It is assumed that this feature is automatically installed if required by the parent feature (SDK), however including it within the AddLocal command line parameter list doesn’t seem to cause any issues.