XML formatting of an entryRelationship with a set type code but null flavour
fieranmason@gmail.com
#1 Posted : Tuesday, October 1, 2013 2:32:11 PM(UTC)
Rank: Member

Groups: Registered
Joined: 7/8/2013(UTC)
Posts: 22
Points: 66
Location: University of Victoria

Thanks: 5 times
Was thanked: 0 time(s) in 0 post(s)
Hello Everest,

We are observing a behaviour of the XML formatter which seems odd to us.

Our schema includes the following line under POCD_MT000040.EntryRelationship

<xs:attribute name="typeCode" type="x_ActRelationshipEntryRelationship" use="required"/>

When we construct an entry relationship with a null flavour, but also include the type code, the type code is not expressed when we publish the document with the xml formatter. This causes us to fail schema validation, though we pass everest validation.

Sample output

<hl7:entryRelationship nullFlavor="UNK" />

Is there a way we can force the inclusion of the typecode in spite of the nullflavour?

Thanks
Fieran



justin.fyfe1
#2 Posted : Tuesday, October 8, 2013 11:34:01 AM(UTC)

Rank: Administration

Medals: Mobile Tech Grasshopper: Mobile Tech GrasshopperHealth Informatics MVP

Groups: Registered, Administrators
Joined: 7/22/2010(UTC)
Posts: 96
Points: 297
Man
Location: Hamilton, ON

Thanks: 2 times
Was thanked: 17 time(s) in 17 post(s)
Hi Fieran,

The use of xsi:nil="true" is supposed to suppress this validation rule, however I think you were using the SuppressXsiNil flag on your formatter. The new version of Everest (1.2.9) which will be posted later today has a new SuppressNullEnforcement flag which will turn of this behavior.

formatter.Settings |= SettingsType.SuppressNullEnforcement;

is how this would be set.

Cheers
-Justin
fieranmason@gmail.com
#3 Posted : Thursday, October 17, 2013 5:52:29 PM(UTC)
Rank: Member

Groups: Registered
Joined: 7/8/2013(UTC)
Posts: 22
Points: 66
Location: University of Victoria

Thanks: 5 times
Was thanked: 0 time(s) in 0 post(s)
Hi Justin,

Did this change suppress just the validation rule or should it also allow the typeCode to be output in the generated xml file? Our schema requires the typeCode be present when a NullFlavor is set. With the 1.2.9 change, we're still seeing the same xml as in the sample output above. Should the 1.2.9 change have affected the xml?

Thanks,
Fieran Mason-Blakley
Phil Sharp
justin.fyfe1
#4 Posted : Friday, October 18, 2013 12:31:57 AM(UTC)

Rank: Administration

Medals: Mobile Tech Grasshopper: Mobile Tech GrasshopperHealth Informatics MVP

Groups: Registered, Administrators
Joined: 7/22/2010(UTC)
Posts: 96
Points: 297
Man
Location: Hamilton, ON

Thanks: 2 times
Was thanked: 17 time(s) in 17 post(s)
Apologies, there was a (quite severe) bug introduced in 1.2.9 which altered the way that the default processing was handled (and it also didn't handle this correctly). Looking at our unit test log it appears that the particular unit tests intended to catch this weren't written correctly.

I will post 1.2.10 in the morning after the updated code base and unit tests have completed running. This behaves as expected:

Example: SuppressNullEnforcement off
Code:

<structuredBody nullFlavor="NI" xsi:nil="true"/>


Example: SuppressNullEnforcement on

Code:

<structuredBody moodCode="EVN" classCode="DOCBODY" nullFlavor="NI" xsi:nil="true"/>


fieranmason@gmail.com
#5 Posted : Monday, October 21, 2013 6:09:44 PM(UTC)
Rank: Member

Groups: Registered
Joined: 7/8/2013(UTC)
Posts: 22
Points: 66
Location: University of Victoria

Thanks: 5 times
Was thanked: 0 time(s) in 0 post(s)
Hi Justin,

Thanks for the responsiveness in the release of 1.2.10. The output produced by this release is exactly what our implementation guide required.

A quick followup question:
Our implementation guide and our schema both require one of the following act/encounter/observation/observationMedia/organizer/procedure/regionOfInterest/substanceAdministration/supply under an Entry Relationship. When we set the Entry Relationship's nullflavor, the Observation we set under the Entry Relationship is being suppressed. Is this expected behavior by the Everest framework?

Regards
Fieran
justin.fyfe1
#6 Posted : Tuesday, October 22, 2013 2:05:12 PM(UTC)

Rank: Administration

Medals: Mobile Tech Grasshopper: Mobile Tech GrasshopperHealth Informatics MVP

Groups: Registered, Administrators
Joined: 7/22/2010(UTC)
Posts: 96
Points: 297
Man
Location: Hamilton, ON

Thanks: 2 times
Was thanked: 17 time(s) in 17 post(s)
Hi Fieran,

The Everest behavior is technically following HL7v3 messaging guidelines which state that when a traversable RMIM association's null flavor is set a nullFlavor attribute and xsi:nil="true" must be emitted. Technically no child elements may appear when xsi:nil is true. At least that is the default behavior of Everest.

However, the new flags I've added to Everest are intended to behave as expected for the majority of CDA schemas. I believe the child element suppression is still being followed, I believe what the real behavior should be is:

Default:
- Emit xsi:nil and NullFlavor (valid according to XML Schema)

SuppressNullEnforcement:
- Emit xsi:nil, NullFlavor and any structural attributes (attributes)

SuppressNullEnforcement | SuppressXsiNil:
- Emit NullFlavor and all structural, non structural and traversable associations

I will update Everest to behave like this as it is technically correct behavior.
psharp
#7 Posted : Monday, December 23, 2013 3:37:10 PM(UTC)
Rank: Member

Groups: Registered
Joined: 11/12/2013(UTC)
Posts: 19
Points: 57

Thanks: 3 times
Was thanked: 0 time(s) in 0 post(s)
Hey Justin,

Any news on this update to change the flag behavior to output the structures underneath a nullflavored attribute?

Thanks
justin.fyfe1
#8 Posted : Thursday, January 16, 2014 1:06:22 PM(UTC)

Rank: Administration

Medals: Mobile Tech Grasshopper: Mobile Tech GrasshopperHealth Informatics MVP

Groups: Registered, Administrators
Joined: 7/22/2010(UTC)
Posts: 96
Points: 297
Man
Location: Hamilton, ON

Thanks: 2 times
Was thanked: 17 time(s) in 17 post(s)
I have been looking into this, and it appears to be fixed however I'm not sure if it made it into the 1.2.10 release. I will repackage a new release with this and other bug fixes and post as 1.2.12 (1.2.11 unfortunately cannot be released due to our code signing certificate being expired).

Thanks
-Justin
1 user thanked justin.fyfe1 for this useful post.
psharp on 1/21/2014(UTC)
psharp
#9 Posted : Tuesday, January 21, 2014 6:02:22 PM(UTC)
Rank: Member

Groups: Registered
Joined: 11/12/2013(UTC)
Posts: 19
Points: 57

Thanks: 3 times
Was thanked: 0 time(s) in 0 post(s)
Hi Justin,

Just giving 1.2.12 a spin now.

Seeing a little bit of oddness with regards to a nullFlavor on an EntryRelationship level.

If I don't nullFlavor the entryRelationship but I have everything else under it nullFlavored out I get the following xml:
<hl7:entryRelationship typeCode="SUBJ" contextConductionInd="true">
<hl7:observation nullFlavor="NI" classCode="OBS" moodCode="EVN">
<hl7:code code="CLINSTAT" codeSystem="2.16.840.1.113883.3.3068.10.6.3" codeSystemName="ObservationType-CA-Pending" />
<hl7:text nullFlavor="NI" />
<hl7:value xsi:type="CD" nullFlavor="NI" />
</hl7:observation>
</hl7:entryRelationship>

This results in a somewhat expected validation warning:
The context is null however SuppressNullEnforcement and SuppressXsiNil are set, therefore elements will be graphed. This is not necessarily HL7v3 compliant

When I add a nullFlavor to the entryRelationship level I get the following xml:
<hl7:entryRelationship nullFlavor="NI" typeCode="SUBJ" contextConductionInd="true" />

Seems the structure isn't output in the case of the entryRelationship being nullFlavored out.

On a side note, how is this usually handled? The 2nd line without structure passes Everest validation however if I do xsd validation I get:
The element 'entryRelationship' in namespace 'urn:hl7-org:v3' has incomplete content. List of possible elements expected: 'realmCode, typeId, templateId, sequenceNumber, seperatableInd, act, encounter, observation, observationMedia, organizer, procedure, regionOfInterest, substanceAdministration, supply' in namespace 'urn:hl7-org:v3'.

The requirement to cascade nullFlavors to subitems, and setting up their structure, has been causing me some headaches, but it seems that without doing it you can't be compliant with the schema.

Thanks,
Phil
justin.fyfe1
#10 Posted : Tuesday, January 21, 2014 6:52:51 PM(UTC)

Rank: Administration

Medals: Mobile Tech Grasshopper: Mobile Tech GrasshopperHealth Informatics MVP

Groups: Registered, Administrators
Joined: 7/22/2010(UTC)
Posts: 96
Points: 297
Man
Location: Hamilton, ON

Thanks: 2 times
Was thanked: 17 time(s) in 17 post(s)
Hi Phil,

The first warning is supposed to occur. Technically in HL7v3 when a traversable association is null flavoured it is supposed to have an xsi:nil attribute, this is the most correct v3 representation:

Code:

<hl7:entryRelationship typeCode="SUBJ" contextConductionInd="true">
<hl7:observation nullFlavor="NI" xsi:nil="true" />
</hl7:entryRelationship>


The nullFlavor lets a v3 processor know there is no content and xsi:nil lets an XML processor know that there is no content. When setting SuppressXsiNil and SuppressNullEnforcement Everest will output a non-compliant XML structure as the XML handling (xsi:nil) and HL7v3 null handling are suppressed. If you remove the SuppressXsiNil and SuppressNullEnforcement options from your settings then Everest should output the correct representation of:

Code:

<hl7:entryRelationship xsi:nil="true" nullFlavor="NI" />


As xsi:nil will let the XML validation process know that the element is nil.

We've been able to pass the IHE External Validation Service (EVS) CDA check with neither SuppressXsiNil or SuppressNullEnforcement enabled. Can you try it without these set and let me know if this fixes the formatting?

Cheers
-Justin
psharp
#11 Posted : Tuesday, January 21, 2014 7:44:37 PM(UTC)
Rank: Member

Groups: Registered
Joined: 11/12/2013(UTC)
Posts: 19
Points: 57

Thanks: 3 times
Was thanked: 0 time(s) in 0 post(s)
Hi Justin,

I removed the 2 flags and indeed got the expected output:
Code:
<hl7:entryRelationship xsi:nil="true" nullFlavor="NI" />


However then I'm back to the issue with xsd validation and the required elements for entryRelationship not being present and my spec's requirements to show subitems and cascade nullflavors to the subitems. It seems odd to me to not have any structure under a nullFlavor since you wouldn't know the type of entryRelationship that contains the nullFlavor from just the line above.

Thanks,
Phil

justin.fyfe1
#12 Posted : Wednesday, January 22, 2014 11:32:06 AM(UTC)

Rank: Administration

Medals: Mobile Tech Grasshopper: Mobile Tech GrasshopperHealth Informatics MVP

Groups: Registered, Administrators
Joined: 7/22/2010(UTC)
Posts: 96
Points: 297
Man
Location: Hamilton, ON

Thanks: 2 times
Was thanked: 17 time(s) in 17 post(s)
Hmm, ok I see.

Technically the behavior is correct, however if your spec requires cascading of null flavours you'll have to write some code to do this as it isn't built in functionality. The code is rather simple and I've posted it here.

First, write an Extension method that adds a "CascadeNullFlavors" method to the IImplementsNullFlavor interface:

Code:

public static class ExtensionMethods
{
    public static void CascadeNullFlavor(this IImplementsNullFlavor me, NullFlavor flavor)
    {
        me.NullFlavor = flavor;
        foreach (var propertyInfo in me.GetType().GetProperties(BindingFlags.Public | BindingFlags.Instance))
        {
               
            Type propertyType = propertyInfo.PropertyType;
            if(typeof(IList).IsAssignableFrom(propertyType))  // correct for lists
                propertyType = propertyType.GetGenericArguments()[0];

            // Only IImpleentsNullFlavor should be allowed
            if (!typeof(IImplementsNullFlavor).IsAssignableFrom(propertyType))
                continue;
           
            // Make some intelligent decisions about populating
            var propertyAtt = propertyInfo.GetCustomAttributes(typeof(PropertyAttribute), true);
            if (propertyAtt.Length > 0)
            {
                var strongPropertyAtt = propertyAtt[0] as PropertyAttribute;
                // Don't set a null flavor on structural (attributes)
                if (strongPropertyAtt.PropertyType == PropertyAttribute.AttributeAttributeType.Structural)
                    continue;
                if (strongPropertyAtt.Conformance == PropertyAttribute.AttributeConformanceType.Optional)
                    continue;

            }
               
            // is this a list or IImplementsNullFlavor
            var constructorInfo = propertyType.GetConstructor(Type.EmptyTypes);
            if (constructorInfo == null) continue; // sanity check, can't create
               
            // Construct an instance
            var instance = constructorInfo.Invoke(null) as IImplementsNullFlavor;

            // Don't cascade down data types
            if (instance is IAny)
                instance.NullFlavor = flavor;
            else
                instance.CascadeNullFlavor(flavor);

            // SEt if not a list
            if (propertyInfo.PropertyType == propertyType)
                propertyInfo.SetValue(me, instance, null);
            else // Add to list if not
                (propertyInfo.GetValue(me, null) as IList).Add(instance);

        }
    }
}


Then use this method to set a null flavour, then use the formatter with NullEnforcement and XSINil suppressed:

Code:

var statement = new Observation();
statement.CascadeNullFlavor(NullFlavor.NoInformation);
doc.Component.GetBodyChoiceIfStructuredBody().Component[0].Section.Entry[0].SetClinicalStatement(statement);

// Graph
XmlIts1Formatter fmtr = new XmlIts1Formatter();
fmtr.GraphAides.Add(new ClinicalDocumentDatatypeFormatter());
fmtr.Settings |= SettingsType.SuppressNullEnforcement | SettingsType.SuppressXsiNil;
using(XmlWriter xw = XmlWriter.Create("test.xml"))
    fmtr.Graph(xw, doc);


This will result in what I think you're looking for which is :

Code:
<ClinicalDocument xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" classCode="DOCCLIN" moodCode="EVN" xmlns="urn:hl7-org:v3">
  <component contextConductionInd="true">
    <structuredBody classCode="DOCBODY" moodCode="EVN">
      <component contextConductionInd="true">
        <section classCode="DOCSECT" moodCode="EVN">
          <entry contextConductionInd="true">
            <observation nullFlavor="NI" classCode="OBS">
              <code nullFlavor="NI" />
            </observation>
          </entry>
        </section>
      </component>
    </structuredBody>
  </component>
</ClinicalDocument>


Note that this code will only cascade null flavours if the property is :

1. Not structural (i.e. not an attribute in XML)
2. Not optional


Cheers
-Justin
justin.fyfe1
#13 Posted : Wednesday, January 22, 2014 11:46:56 AM(UTC)

Rank: Administration

Medals: Mobile Tech Grasshopper: Mobile Tech GrasshopperHealth Informatics MVP

Groups: Registered, Administrators
Joined: 7/22/2010(UTC)
Posts: 96
Points: 297
Man
Location: Hamilton, ON

Thanks: 2 times
Was thanked: 17 time(s) in 17 post(s)
Also, many times CDA specs will have more restrictive multiplicity placed on certain items.

The CDA classes in Everest are based on the generic CDA MIF from HL7 which has almost everything marked as "Optional" which means that your schema still might not validate. There are a few things that you can do implement this.

1. You can profile each Everest class to match your schema, this is done by setting the Conformance property on the [Property] attribute to Mandatory, example:

Code:

[Structure(Model = "POCD_MT000040", Name = "Observation", StructureType = StructureAttribute.StructureAttributeType.MessageType)]
public class ObservationMyProfile : Observation
{
    [Property(Name = "value", PropertyType = PropertyAttribute.AttributeAttributeType.NonStructural, Conformance = PropertyAttribute.AttributeConformanceType.Mandatory)]
    public new ANY Value { get; set; }
}


2. You can have your cascade null flavours read in a CDA schema and locate the appropriate element within the class and use the schema's multiplicity
3. I am working on a profiling tool for Everest that will generate a series of overridden classes from a CDA.xsd file. This is currently being developed and might take a little while before it appears in Everest as it is technically a planned 1.4 feature.

Thanks
-Justin
psharp
#14 Posted : Wednesday, January 22, 2014 2:41:43 PM(UTC)
Rank: Member

Groups: Registered
Joined: 11/12/2013(UTC)
Posts: 19
Points: 57

Thanks: 3 times
Was thanked: 0 time(s) in 0 post(s)
Thanks Justin,

I forwarded this thread on to the designers of my spec and it sounds like my requirements may be changing to make us more compliant. Hopefully I'll be able to have this all resolved in the next couple days and won't have to create an invalid xml file to meet spec requirements. It's great that it's so easy to do though if it does become required.
psharp
#15 Posted : Thursday, January 23, 2014 1:13:03 PM(UTC)
Rank: Member

Groups: Registered
Joined: 11/12/2013(UTC)
Posts: 19
Points: 57

Thanks: 3 times
Was thanked: 0 time(s) in 0 post(s)
Hey Justin,

The response I got unfortunately was:
Quote:
The following code <hl7:entryRelationship xsi:nil="true" nullFlavor="NI" /> is not allowed.


Looks like I'll need to do that cascade structure afterall. I'll give your code a go sometime soon. It looks straightforward enough.
psharp
#16 Posted : Wednesday, March 5, 2014 12:17:26 AM(UTC)
Rank: Member

Groups: Registered
Joined: 11/12/2013(UTC)
Posts: 19
Points: 57

Thanks: 3 times
Was thanked: 0 time(s) in 0 post(s)
Hey Justin,

I'm seeing a little wierdness with this. I'm using Everest 1.2.13. I'm unsure if this is new to this version or not.

Below is how I'm putting my document into a FileStream:
Code:

            using (FileStream stream = File.Open(cdaFileInfo.FullName, FileMode.CreateNew))
            {
                XmlIts1Formatter fmtr = new XmlIts1Formatter();
                fmtr.ValidateConformance = false;
                fmtr.GraphAides.Add(new ClinicalDocumentDatatypeFormatter());
                fmtr.Settings |= SettingsType.SuppressXsiNil;
                fmtr.Settings |= SettingsType.SuppressNullEnforcement;
                fmtr.Settings |= SettingsType.AlwaysCheckForOverrides;
                fmtr.RegisterXSITypeName("POCD_MT000040UV.Observation", typeof(ObservationWithConfidentialityCode));

                using (XmlWriter writer = XmlWriter.Create(stream, new XmlWriterSettings() { Indent = true }))
                using (XmlStateWriter xsw = new XmlStateWriter(writer))
                {
                    xsw.WriteStartElement("hl7", "ClinicalDocument", "urn:hl7-org:v3");
                    xsw.WriteAttributeString("xmlns", "xsi", null, XmlIts1Formatter.NS_XSI);
                    xsw.WriteAttributeString("schemaLocation", XmlIts1Formatter.NS_XSI, @"urn:hl7-org:v3 Schemas/CDA-PITO-E2E.xsd");
                    xsw.WriteAttributeString("xmlns", null, null, @"urn:hl7-org:v3");
                    xsw.WriteAttributeString("xmlns", "hl7", null, @"urn:hl7-org:v3");
                    xsw.WriteAttributeString("xmlns", "e2e", null, @"http://standards.pito.bc.ca/E2E-DTC/cda");
                    xsw.WriteAttributeString("xmlns", "xs", null, @"http://www.w3.org/2001/XMLSchema");

                    IFormatterGraphResult result = fmtr.Graph(xsw, clinicalDocument);
                    xsw.WriteEndElement(); // clinical document
                    xsw.Flush();
                }
            }


When I do sometimes I get the following result:
Quote:
<entryRelationship nullFlavor="UNK" typeCode="COMP" contextConductionInd="true">
<observation nullFlavor="UNK" classCode="OBS" moodCode="EVN">
<code code="ICD9CODE" codeSystem="2.16.840.1.113883.3.3068.10.6.3" codeSystemName="ObservationType-CA-Pending" />
</observation>
</entryRelationship>


And then sometimes I get the following:
Quote:
<entryRelationship nullFlavor="UNK" typeCode="COMP" contextConductionInd="true" />



Even stranger is I've had it show the subitems for one nullFlavored entryrelationship, but not another:
Quote:
<entryRelationship nullFlavor="UNK" typeCode="COMP" contextConductionInd="true" />

Quote:
<entryRelationship nullFlavor="NI" typeCode="COMP" contextConductionInd="true">
<procedure nullFlavor="NI" classCode="PROC" moodCode="EVN">
<effectiveTime nullFlavor="NI" />
</procedure>
</entryRelationship>



With a small document it tends to not show the subitems. With a larger document it tends to show the subitems.
justin.fyfe1
#17 Posted : Wednesday, March 5, 2014 7:22:31 AM(UTC)

Rank: Administration

Medals: Mobile Tech Grasshopper: Mobile Tech GrasshopperHealth Informatics MVP

Groups: Registered, Administrators
Joined: 7/22/2010(UTC)
Posts: 96
Points: 297
Man
Location: Hamilton, ON

Thanks: 2 times
Was thanked: 17 time(s) in 17 post(s)
Its probably a bug with the CodeDom formatter. I will run this through some unit tests today and track down the issue. For now; if you set the settings to DefaultUniprocessor and then adding the suppression flags.

Cheers
-Justin
justin.fyfe1
#18 Posted : Wednesday, March 5, 2014 10:06:44 AM(UTC)

Rank: Administration

Medals: Mobile Tech Grasshopper: Mobile Tech GrasshopperHealth Informatics MVP

Groups: Registered, Administrators
Joined: 7/22/2010(UTC)
Posts: 96
Points: 297
Man
Location: Hamilton, ON

Thanks: 2 times
Was thanked: 17 time(s) in 17 post(s)
Ok, I found the issue and have corrected it. It turns out it was the reflection formatter that was the cause of this. If you want to workaround until I release the fix you can do this:

Code:

                fmtr.Settings = SettingsType.DefaultLegacy;
                fmtr.Settings |= SettingsType.SuppressXsiNil;
                fmtr.Settings |= SettingsType.SuppressNullEnforcement;
                fmtr.Settings |= SettingsType.AlwaysCheckForOverrides;


And the issue won't appear. Otherwise I will re-run the unit tests and will post a 1.2.14 installer containing the fix in a few hours.
1 user thanked justin.fyfe1 for this useful post.
psharp on 3/5/2014(UTC)
psharp
#19 Posted : Wednesday, March 5, 2014 12:58:29 PM(UTC)
Rank: Member

Groups: Registered
Joined: 11/12/2013(UTC)
Posts: 19
Points: 57

Thanks: 3 times
Was thanked: 0 time(s) in 0 post(s)
Workarounds like this don't bother me at all. No rush on 1.2.14. Will give it a try shortly. Much thanks.
Users browsing this topic
Guest (6)
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

SoClean Theme By Jaben Cargman (Tiny Gecko)
Powered by YAF 1.9.4 | YAF © 2003-2010, Yet Another Forum.NET
This page was generated in 0.264 seconds.