I am trying to load a packed decimal field value from a flat file (fixed length) to a SQL Server table by using SSIS 2012.
The schema of this field is descibed as:
Field Name: Field10
From Position: 20
To Position: 25
Length: 6
Decimal Positions: 2
Data Type: PACKED
Addtional Notes: Actually this field is 11 digits with 2 decimal positions-PACKED. Ensure the 'To' positions are calculated correctly with the cell formula by changing
the data length to actual positions-UNPACKED data.
So far I have used the information and techniques explained in the following links.
http://stackoverflow.com/questions/795523/interpreting-comp-3-packed-decimal-fields-into-numeric-values
http://www.mssqltips.com/sqlservertip/3459/importing-mainframe-data-including-packed-numbers-with-sql-server-integration-services/
http://mikedavissql.com/2011/08/31/convert-ebcdic-to-ascii-in-ssis/
http://www.microsoft.com/en-us/download/details.aspx?id=20397
I added a flat file connection manager and set the 'UseBinaryFormat' os this column to true in the flat file source. I have marked this filed as byte array in the connection manager.
This field value displays as a six byte array in the data viewer for all records in the file.
The problem is - my SSIS transformations are rejecting nearly two third of the records. My best guess is that a packed decimal field, when read as an array of bytes will
have values between 0 and 9 except for the last (right most) sign nibble. The last sign nibble can be C or D or F.
When I use a data viewer to read the data and byte array, the first few records have the value of this field as:
0x00 0x00 0x00 0x03 0x17 0x5F
0x00 0x00 0x00 0x88 0xC8 0x29
0x00 0x00 0x00 0x00 0x6F 0x29
0x00 0x00 0x00 0x01 0xD8 0x5F
0x00 0x00 0x01 0xC9 0xC1 0xB1
0x00 0x00 0x00 0x19 0x11 0xA4
0x00 0x00 0x00 0x03 0x17 0x3F
0x00 0x00 0x00 0x88 0xC8 0xAC
0x00 0x00 0x00 0x00 0x6F 0xAC
0x00 0x00 0x00 0x01 0xD8 0x3F
0x00 0x00 0x02 0x7F 0x20 0x0D
0x00 0x00 0x00 0x02 0x72 0x29
0x00 0x00 0x00 0x02 0x90 0xAC
0x00 0x00 0x00 0x16 0x84 0x0D
0x00 0x00 0x00 0x16 0x84 0x0D
0x00 0x00 0x00 0x16 0x84 0x0F
0x00 0x00 0x00 0x16 0x84 0x0F
0x00 0x00 0x01 0x64 0xEF 0x1A
0x00 0x00 0x94 0xE4 0x00 0x7C
0x00 0x00 0x20 0x80 0x93 0xAC
0x00 0x00 0x00 0xEF 0x65 0x1A
0x00 0x00 0x13 0x71 0x60 0x1A
As far as I know, each of these bytes should end with C or D or F. All other characters in the byte array should have a digit between 0 and 9. Is my understanding correct?
I am in the process of reaching out to the people who provided those flat files but just want to reach out to the community to see if someone can point out if I am missing something.
All the fields other than this field are readable to me by directly opening the file and are loading just fine. So this looks like an ASCII file with one field which is packed decimal (may be Field10 is encoded differently as it seems filled with weird characters in the file).
Thanks in advance for your help.
The schema of this field is descibed as:
Field Name: Field10
From Position: 20
To Position: 25
Length: 6
Decimal Positions: 2
Data Type: PACKED
Addtional Notes: Actually this field is 11 digits with 2 decimal positions-PACKED. Ensure the 'To' positions are calculated correctly with the cell formula by changing
the data length to actual positions-UNPACKED data.
So far I have used the information and techniques explained in the following links.
http://stackoverflow.com/questions/795523/interpreting-comp-3-packed-decimal-fields-into-numeric-values
http://www.mssqltips.com/sqlservertip/3459/importing-mainframe-data-including-packed-numbers-with-sql-server-integration-services/
http://mikedavissql.com/2011/08/31/convert-ebcdic-to-ascii-in-ssis/
http://www.microsoft.com/en-us/download/details.aspx?id=20397
I added a flat file connection manager and set the 'UseBinaryFormat' os this column to true in the flat file source. I have marked this filed as byte array in the connection manager.
This field value displays as a six byte array in the data viewer for all records in the file.
The problem is - my SSIS transformations are rejecting nearly two third of the records. My best guess is that a packed decimal field, when read as an array of bytes will
have values between 0 and 9 except for the last (right most) sign nibble. The last sign nibble can be C or D or F.
When I use a data viewer to read the data and byte array, the first few records have the value of this field as:
0x00 0x00 0x00 0x03 0x17 0x5F
0x00 0x00 0x00 0x88 0xC8 0x29
0x00 0x00 0x00 0x00 0x6F 0x29
0x00 0x00 0x00 0x01 0xD8 0x5F
0x00 0x00 0x01 0xC9 0xC1 0xB1
0x00 0x00 0x00 0x19 0x11 0xA4
0x00 0x00 0x00 0x03 0x17 0x3F
0x00 0x00 0x00 0x88 0xC8 0xAC
0x00 0x00 0x00 0x00 0x6F 0xAC
0x00 0x00 0x00 0x01 0xD8 0x3F
0x00 0x00 0x02 0x7F 0x20 0x0D
0x00 0x00 0x00 0x02 0x72 0x29
0x00 0x00 0x00 0x02 0x90 0xAC
0x00 0x00 0x00 0x16 0x84 0x0D
0x00 0x00 0x00 0x16 0x84 0x0D
0x00 0x00 0x00 0x16 0x84 0x0F
0x00 0x00 0x00 0x16 0x84 0x0F
0x00 0x00 0x01 0x64 0xEF 0x1A
0x00 0x00 0x94 0xE4 0x00 0x7C
0x00 0x00 0x20 0x80 0x93 0xAC
0x00 0x00 0x00 0xEF 0x65 0x1A
0x00 0x00 0x13 0x71 0x60 0x1A
As far as I know, each of these bytes should end with C or D or F. All other characters in the byte array should have a digit between 0 and 9. Is my understanding correct?
I am in the process of reaching out to the people who provided those flat files but just want to reach out to the community to see if someone can point out if I am missing something.
All the fields other than this field are readable to me by directly opening the file and are loading just fine. So this looks like an ASCII file with one field which is packed decimal (may be Field10 is encoded differently as it seems filled with weird characters in the file).
Thanks in advance for your help.