C #: convert COMP-3 with decimal precision to human readable value

I have a series of flat ASCII files coming from a mainframe that will be processed by a C # application. A new feed was introduced with the Packed Decimal (COMP-3) field, which must be converted to a numerical value.

Files are transferred via FTP using ASCII transfer mode. I am worried that the binary field may contain what would be interpreted as very low ASCII codes or control characters instead of a value. Or, even worse, it may be lost in the FTP process.

What more, fields are read as strings. I may have the flexibility to work on this part (i.e., some kind of stream), but the business will give me a rollback.

The requirement to read "Convert from HEX to ASCII", but it is clear that they did not give the correct values. Any help would be greatly appreciated; it does not necessarily depend on the language, if you can explain the logic of the conversion process.

+6
c # flat-file ibm-midrange comp-3
source share
6 answers

First of all, you must fix the end of line feed (EOL) problems that will be caused by ASCII transfer mode. You are absolutely right to worry about data corruption when the BCD values ​​match the EOL characters. The worst aspect of this problem is that it will occur rarely and unexpectedly.

The best solution is to change the transfer mode to BIN. This is necessary since the data you are transmitting is binary. If you cannot use the correct FTP transfer mode, you can undo the ASCII mode corruption in the code. All you have to do is convert the \ r \ n pairs back to \ n. If I were you, I would make sure it is well tested.

After solving the EOL problem, converting COMP-3 is quite difficult. I was able to find this article in the MS Knowledge Base with sample code in BASIC. See below for the VB.NET port of this code.

Since you are dealing with COMP-3 values, the file format you are reading almost certainly has fixed record sizes with fixed field lengths. If I were you, I would get my hands on the file format specification before moving on with this. You must use BinaryReader to work with this data. If someone starts from this moment, I would leave. Let them find someone else to pamper their stupidity.

Here is the VB.NET port of the BASIC sample code. I have not tested this because I do not have access to the COMP-3 file. If this does not work, I will return to the original MS code sample for guidance or to the links in other answers to this question.

Imports Microsoft.VisualBasic Module Module1 'Sample COMP-3 conversion code 'Adapted from http://support.microsoft.com/kb/65323 'This code has not been tested Sub Main() Dim Digits%(15) 'Holds the digits for each number (max = 16). Dim Basiceqv#(1000) 'Holds the Basic equivalent of each COMP-3 number. 'Added to make code compile Dim MyByte As Char, HighPower%, HighNibble% Dim LowNibble%, Digit%, E%, Decimal%, FileName$ 'Clear the screen, get the filename and the amount of decimal places 'desired for each number, and open the file for sequential input: FileName$ = InputBox("Enter the COBOL data file name: ") Decimal% = InputBox("Enter the number of decimal places desired: ") FileOpen(1, FileName$, OpenMode.Binary) Do Until EOF(1) 'Loop until the end of the file is reached. Input(1, MyByte) If MyByte = Chr(0) Then 'Check if byte is 0 (ASC won't work on 0). Digits%(HighPower%) = 0 'Make next two digits 0. Increment Digits%(HighPower% + 1) = 0 'the high power to reflect the HighPower% = HighPower% + 2 'number of digits in the number 'plus 1. Else HighNibble% = Asc(MyByte) \ 16 'Extract the high and low LowNibble% = Asc(MyByte) And &HF 'nibbles from the byte. The Digits%(HighPower%) = HighNibble% 'high nibble will always be a 'digit. If LowNibble% <= 9 Then 'If low nibble is a 'digit, assign it and Digits%(HighPower% + 1) = LowNibble% 'increment the high HighPower% = HighPower% + 2 'power accordingly. Else HighPower% = HighPower% + 1 'Low nibble was not a digit but a Digit% = 0 '+ or - signals end of number. 'Start at the highest power of 10 for the number and multiply 'each digit by the power of 10 place it occupies. For Power% = (HighPower% - 1) To 0 Step -1 Basiceqv#(E%) = Basiceqv#(E%) + (Digits%(Digit%) * (10 ^ Power%)) Digit% = Digit% + 1 Next 'If the sign read was negative, make the number negative. If LowNibble% = 13 Then Basiceqv#(E%) = Basiceqv#(E%) - (2 * Basiceqv#(E%)) End If 'Give the number the desired amount of decimal places, print 'the number, increment E% to point to the next number to be 'converted, and reinitialize the highest power. Basiceqv#(E%) = Basiceqv#(E%) / (10 ^ Decimal%) Print(Basiceqv#(E%)) E% = E% + 1 HighPower% = 0 End If End If Loop FileClose() 'Close the COBOL data file, and end. End Sub End Module 
+6
source share

I watched posts on numerous tips regarding converting Comp-3 BCD data from "old" mainframes to something that can be used in C #. Firstly, I would like to say that I am less than enthusiastic about the answers that were received by some of these posts, especially those that said in essence: "Why are you bothering us with these messages other than C # / C + + ", as well as" If you need an answer about some COBOL agreement, why don't you go to a COBOL-oriented site. " For me, this is a complete BS, because for software developers, unfortunately, it will probably take many years to figure out how to deal with some of these problems that exist in THE REAL WORLD. Thus, even if I slam this post for the following code, I am going to share with you the REAL WORLD experience that I had to deal with regarding COMP-3 / EBCDIC conversion (and yes, I am the one who talks about “floppy disks, paper tapes, discs, etc. - I have been a software engineer since 1979).

First you need to understand that any file that you read from an outdated system of the main frame, for example IBM, will present you data in EBCDIC format and to convert any of this data to a C # / C ++ string that you may allow you have to use correct translation of the code page to receive data in ASCII format. A good example of how to handle this would be:

StreamReader readFile = new StreamReader (path, encoding .GetEncoding (037); // 037 = EBCDIC to ASCII conversion.

This ensures that everything you read from this stream is then converted to ASCII and can be used in string format. This includes the “Zoned Decimal” (Fig. 9) and “Text” (Fig. X) fields as announced by COBOL. However, this does not necessarily convert the COMP-3 fields to the correct “binary” equivalent when reading into a char [] or byte [] array. To do this, the only way you are ever going to translate it correctly (even using the code pages UTF-8, UTF-16, Default or something else), you need to open the file as follows:

FileStream fileStream = new FileStream (path, FIleMode.Open, FIleAccess.Read, FileShare.Read);

Of course, the "FileShare.Read" option is "optional."

When you isolate the field that you want to convert to a decimal value (and then, if necessary, to an ASCII string), you can use the following code - and this was mainly stolen from MicroSoft's “UnpackDecimal” message, which you can get:

http://www.microsoft.com/downloads/details.aspx?familyid=0e4bba52-cc52-4d89-8590-cda297ff7fbd&displaylang=en

I highlighted (I think) what are the most important parts of this logic and combined it into two methods that you can do with what you want. For my purposes, I decided to leave this as a decimal value that I could do with what I wanted. Basically, the method is called “unzip”, and you pass it a byte [] array (no more than 12 bytes), and the scale as int, which is the number of decimal places that you want to return in decimal value. I hope this works for you, as it does for me.

  private Decimal Unpack(byte[] inp, int scale) { long lo = 0; long mid = 0; long hi = 0; bool isNegative; // this nybble stores only the sign, not a digit. // "C" hex is positive, "D" hex is negative, and "F" hex is unsigned. switch (nibble(inp, 0)) { case 0x0D: isNegative = true; break; case 0x0F: case 0x0C: isNegative = false; break; default: throw new Exception("Bad sign nibble"); } long intermediate; long carry; long digit; for (int j = inp.Length * 2 - 1; j > 0; j--) { // multiply by 10 intermediate = lo * 10; lo = intermediate & 0xffffffff; carry = intermediate >> 32; intermediate = mid * 10 + carry; mid = intermediate & 0xffffffff; carry = intermediate >> 32; intermediate = hi * 10 + carry; hi = intermediate & 0xffffffff; carry = intermediate >> 32; // By limiting input length to 14, we ensure overflow will never occur digit = nibble(inp, j); if (digit > 9) { throw new Exception("Bad digit"); } intermediate = lo + digit; lo = intermediate & 0xffffffff; carry = intermediate >> 32; if (carry > 0) { intermediate = mid + carry; mid = intermediate & 0xffffffff; carry = intermediate >> 32; if (carry > 0) { intermediate = hi + carry; hi = intermediate & 0xffffffff; carry = intermediate >> 32; // carry should never be non-zero. Back up with validation } } } return new Decimal((int)lo, (int)mid, (int)hi, isNegative, (byte)scale); } private int nibble(byte[] inp, int nibbleNo) { int b = inp[inp.Length - 1 - nibbleNo / 2]; return (nibbleNo % 2 == 0) ? (b & 0x0000000F) : (b >> 4); } 

If you have any questions, post them here because I suspect that I am going to get a “blazing” one, like everyone who decided to post questions related to today's issues ...

Thanks, John is an elder.

+6
source share

If the source data was in EBCDIC, your COMP-3 field is distorted. The FTP process has converted EBCDIC to ASCII byte values ​​in the COMP-3 field, which is not what you want. To fix this, you can:

1) Use the BINARY mode for transmission to get the original EBCDIC data. Then you convert the COMP-3 field to a number and convert any other EBCDIC text to an ASCII record. A packed field stores each digit in a nibble with the lower half byte as a character (F is positive, and other values, usually D or E, are negative). Storing 123.4 in PIC 999.99 USING COMP-3 will be X'01234F '(three bytes) and -123 in the same field X'01230D'.

2) Ask the sender to convert the field to USAME IS DISPLAY SIGN LEADING (or TRAILING). This saves the number as an EBCDIC signed digit string as a separate negative (-) or blank character. All digits and characters are correctly translated to their ASCII equivalent when transmitting via FTP.

+3
source share

I apologize if I am leaving the base, but maybe this sample code that I put in here may help you. This came from VBRocks ...

 Imports System Imports System.IO Imports System.Text Imports System.Text.Encoding '4/20/07 submission includes a line spacing addition when a control character is used: ' The line spacing is calculated off of the 3rd control character. ' ' Also includes the 4/18 modification of determining end of file. '4/26/07 submission inclues an addition of 6 to the record length when the 4th control ' character is an 8. This is because these records were being truncated. 'Authored by Gary A. Lima, aka. VBRocks ''' <summary> ''' Translates an EBCDIC file to an ASCII file. ''' </summary> ''' <remarks></remarks> Public Class EBCDIC_to_ASCII_Translator #Region " Example" Private Sub Example() 'Set your source file and destination file paths Dim sSourcePath As String = "c:\Temp\MyEBCDICFile" Dim sDestinationPath As String = "c:\Temp\TranslatedFile.txt" Dim trans As New EBCDIC_to_ASCII_Translator() 'If your EBCDIC file uses Control records to determine the length of a record, then this to True trans.UseControlRecord = True 'If the first record of your EBCDIC file is filler (junk), then set this to True trans.IgnoreFirstRecord = True 'EBCDIC files are written in block lengths, set your block length (Example: 134, 900, Etc.) trans.BlockLength = 900 'This method will actually translate your source file and output it to the specified destination file path trans.TranslateFile(sSourcePath, sDestinationPath) 'Here is a alternate example: 'No Control record is used 'trans.UseControlRecord = False 'Translate the whole file, including the first record 'trans.IgnoreFirstRecord = False 'Set the block length 'trans.BlockLength = 134 'Translate... 'trans.TranslateFile(sSourcePath, sDestinationPath) '*** Some additional methods that you can use are: 'Trim off leading characters from left side of string (position 0 to...) 'trans.LTrim = 15 'Translate 1 EBCDIC character to an ASCII character 'Dim strASCIIChar as String = trans.TranslateCharacter("S") 'Translate an EBCDIC character array to an ASCII string 'trans.TranslateCharacters(chrEBCDICArray) 'Translates an EBCDIC string to an ASCII string 'Dim strASCII As String = trans.TranslateString("EBCDIC String") End Sub #End Region 'Example 'Translate characters from EBCDIC to ASCII Private ASCIIEncoding As Encoding = Encoding.ASCII Private EBCDICEncoding As Encoding = Encoding.GetEncoding(37) 'EBCDIC 'Block Length: Can be fixed (Ex: 134). Private miBlockLength As Integer = 0 Private mbUseControlRec As Boolean = True 'If set to False, will return exact block length Private mbIgnoreFirstRecord As Boolean = True 'Will Ignore first record if set to true (First record may be filler) Private miLTrim As Integer = 0 ''' <summary> ''' Translates SourceFile from EBCDIC to ASCII. Writes output to file path specified by DestinationFile parameter. ''' Set the BlockLength Property to designate block size to read. ''' </summary> ''' <param name="SourceFile">Enter the path of the Source File.</param> ''' <param name="DestinationFile">Enter the path of the Destination File.</param> ''' <remarks></remarks> Public Sub TranslateFile(ByVal SourceFile As String, ByVal DestinationFile As String) Dim iRecordLength As Integer 'Stores length of a record, not including the length of the Control Record (if used) Dim sRecord As String = "" 'Stores the actual record Dim iLineSpace As Integer = 1 'LineSpace: 1 for Single Space, 2 for Double Space, 3 for Triple Space... Dim iControlPosSix As Byte() 'Stores the 6th character of a Control Record (used to calculate record length) Dim iControlRec As Byte() 'Stores the EBCDIC Control Record (First 6 characters of record) Dim bEOR As Boolean 'End of Record Flag Dim bBOF As Boolean = True 'Beginning of file Dim iConsumedChars As Integer = 0 'Stores the number of consumed characters in the current block Dim bIgnoreRecord As Boolean = mbIgnoreFirstRecord 'Ignores the first record if set. Dim ControlArray(5) As Char 'Stores Control Record (first 6 bytes) Dim chrArray As Char() 'Stores characters just after read from file Dim sr As New StreamReader(SourceFile, EBCDICEncoding) Dim sw As New StreamWriter(DestinationFile) 'Set the RecordLength to the RecordLength Property (below) iRecordLength = miBlockLength 'Loop through entire file Do Until sr.EndOfStream = True 'If using a Control Record, then check record for valid data. If mbUseControlRec = True Then 'Read the Control Record (first 6 characters of the record) sr.ReadBlock(ControlArray, 0, 6) 'Update the value of consumed (read) characters iConsumedChars += ControlArray.Length 'Get the bytes of the Control Record Array iControlRec = EBCDICEncoding.GetBytes(ControlArray) 'Set the line spacing (position 3 divided by 64) ' (64 decimal = Single Spacing; 128 decimal = Double Spacing) iLineSpace = iControlRec(2) / 64 'Check the Control record for End of File 'If the Control record has a 8 or 10 in position 1, and a 1 in postion 2, then it is the end of the file If (iControlRec(0) = 8 OrElse iControlRec(0) = 10) AndAlso _ iControlRec(1) = 1 Then If bBOF = False Then Exit Do Else 'The Beginning of file flag is set to true by default, so when the first ' record is encountered, it is bypassed and the bBOF flag is set to False bBOF = False End If 'If bBOF = Fals End If 'If (iControlRec(0) = 8 OrElse 'Set the default value for the End of Record flag to True ' If the Control Record has all zeros, then it True, else False bEOR = True 'If the Control record contains all zeros, bEOR will stay True, else it will be set to False For i As Integer = 0 To 5 If iControlRec(i) > 0 Then bEOR = False Exit For End If 'If iControlRec(i) > 0 Next 'For i As Integer = 0 To 5 If bEOR = False Then 'Convert EBCDIC character to ASCII 'Multiply the 6th byte by 6 to get record length ' Why multiply by 6? Because it works. iControlPosSix = EBCDICEncoding.GetBytes(ControlArray(5)) 'If the 4th position of the control record is an 8, then add 6 ' to the record length to pick up remaining characters. If iControlRec(3) = 8 Then iRecordLength = CInt(iControlPosSix(0)) * 6 + 6 Else iRecordLength = CInt(iControlPosSix(0)) * 6 End If 'Add the length of the record to the Consumed Characters counter iConsumedChars += iRecordLength Else 'If the Control Record had all zeros in it, then it is the end of the Block. 'Consume the remainder of the block so we can continue at the beginning of the next block. ReDim chrArray(miBlockLength - iConsumedChars - 1) 'ReDim chrArray(iRecordLength - iConsumedChars - 1) 'Consume (read) the remaining characters in the block. ' We are not doing anything with them because they are not actual records. 'sr.ReadBlock(chrArray, 0, iRecordLength - iConsumedChars) sr.ReadBlock(chrArray, 0, miBlockLength - iConsumedChars) 'Reset the Consumed Characters counter iConsumedChars = 0 'Set the Record Length to 0 so it will not be processed below. iRecordLength = 0 End If ' If bEOR = False End If 'If mbUseControlRec = True If iRecordLength > 0 Then 'Resize our array, dumping previous data. Because Arrays are Zero (0) based, subtract 1 from the Record length. ReDim chrArray(iRecordLength - 1) 'Read the specfied record length, without the Control Record, because we already consumed (read) it. sr.ReadBlock(chrArray, 0, iRecordLength) 'Copy Character Array to String Array, Converting in the process, then Join the Array to a string sRecord = Join(Array.ConvertAll(chrArray, New Converter(Of Char, String)(AddressOf ChrToStr)), "") 'If the record length was 0, then the Join method may return Nothing If IsNothing(sRecord) = False Then If bIgnoreRecord = True Then 'Do nothing - bypass record 'Reset flag bIgnoreRecord = False Else 'Write the line out, LTrimming the specified number of characters. If sRecord.Length >= miLTrim Then sw.WriteLine(sRecord.Remove(0, miLTrim)) Else sw.WriteLine(sRecord.Remove(0, sRecord.Length)) End If ' If sRecord.Length >= miLTrim 'Write out the number of blank lines specified by the 3rd control character. For i As Integer = 1 To iLineSpace - 1 sw.WriteLine("") Next 'For i As Integer = 1 To iLineSpace End If 'If bIgnoreRecord = True 'Obviously, if we have read more characters from the file than the designated size of the block, ' then subtract the number of characters we have read into the next block from the block size. If iConsumedChars > miBlockLength Then 'If iConsumedChars > iRecordLength Then iConsumedChars = iConsumedChars - miBlockLength 'iConsumedChars = iConsumedChars - iRecordLength End If End If 'If IsNothing(sRecord) = False End If 'If iRecordLength > 0 'Allow computer to process (works in a class module, not in a dll) 'Application.DoEvents() Loop 'Destroy StreamReader (sr) sr.Close() sr.Dispose() 'Destroy StreamWriter (sw) sw.Close() sw.Dispose() End Sub ''' <summary> ''' Translates 1 EBCDIC Character (Char) to an ASCII String ''' </summary> ''' <param name="chr"></param> ''' <returns></returns> ''' <remarks></remarks> Private Function ChrToStr(ByVal chr As Char) As String Dim sReturn As String = "" 'Convert character into byte Dim EBCDICbyte As Byte() = EBCDICEncoding.GetBytes(chr) 'Convert EBCDIC byte to ASCII byte Dim ASCIIByte As Byte() = Encoding.Convert(EBCDICEncoding, ASCIIEncoding, EBCDICbyte) sReturn = Encoding.ASCII.GetString(ASCIIByte) Return sReturn End Function ''' <summary> ''' Translates an EBCDIC String to an ASCII String ''' </summary> ''' <param name="sStringToTranslate"></param> ''' <returns>String</returns> ''' <remarks></remarks> Public Function TranslateString(ByVal sStringToTranslate As String) As String Dim i As Integer = 0 Dim sReturn As New System.Text.StringBuilder() 'Loop through the string and translate each character For i = 0 To sStringToTranslate.Length - 1 sReturn.Append(ChrToStr(sStringToTranslate.Substring(i, 1))) Next Return sReturn.ToString() End Function ''' <summary> ''' Translates 1 EBCDIC Character (Char) to an ASCII String ''' </summary> ''' <param name="sCharacterToTranslate"></param> ''' <returns>String</returns> ''' <remarks></remarks> Public Function TranslateCharacter(ByVal sCharacterToTranslate As Char) As String Return ChrToStr(sCharacterToTranslate) End Function ''' <summary> ''' Translates an EBCDIC Character (Char) Array to an ASCII String ''' </summary> ''' <param name="sCharacterArrayToTranslate"></param> ''' <returns>String</returns> ''' <remarks>Remarks</remarks> Public Function TranslateCharacters(ByVal sCharacterArrayToTranslate As Char()) As String Dim sReturn As String = "" 'Copy Character Array to String Array, Converting in the process, then Join the Array to a string sReturn = Join(Array.ConvertAll(sCharacterArrayToTranslate, _ New Converter(Of Char, String)(AddressOf ChrToStr)), "") Return sReturn End Function ''' <summary> ''' Block Length must be set. You can set the BlockLength for specific block sizes (Ex: 134). ''' Set UseControlRecord = False for files with specific block sizes (Default is True) ''' </summary> ''' <value>0</value> ''' <returns>Integer</returns> ''' <remarks></remarks> Public Property BlockLength() As Integer Get Return miBlockLength End Get Set(ByVal value As Integer) miBlockLength = value End Set End Property ''' <summary> ''' Determines whether a ControlKey is used to calculate RecordLength of valid data ''' </summary> ''' <value>Default value is True</value> ''' <returns>Boolean</returns> ''' <remarks></remarks> Public Property UseControlRecord() As Boolean Get Return mbUseControlRec End Get Set(ByVal value As Boolean) mbUseControlRec = value End Set End Property ''' <summary> ''' Ignores first record if set (Default is True) ''' </summary> ''' <value>Default is True</value> ''' <returns>Boolean</returns> ''' <remarks></remarks> Public Property IgnoreFirstRecord() As Boolean Get Return mbIgnoreFirstRecord End Get Set(ByVal value As Boolean) mbIgnoreFirstRecord = value End Set End Property ''' <summary> ''' Trims the left side of every string the specfied number of characters. Default is 0. ''' </summary> ''' <value>Default is 0.</value> ''' <returns>Integer</returns> ''' <remarks></remarks> Public Property LTrim() As Integer Get Return miLTrim End Get Set(ByVal value As Integer) miLTrim = value End Set End Property End Class 
+2
source share

Some useful links for translating EBCDIC:

Translation table - it is useful to check some values ​​in packed decimal fields: http://www.simotime.com/asc2ebc1.htm

List of code pages in msdn:
http://msdn.microsoft.com/en-us/library/dd317756(VS.85).aspx

And a piece of code to convert byte array fields to C #:

 // 500 is the code page for IBM EBCDIC International System.Text.Encoding enc = new System.Text.Encoding(500); string value = enc.GetString(byteArrayField); 
0
source share

Packed fields are the same in EBCDIC or ASCII. Do not convert EBCDIC to ASCII. In .Net, upload them to byte [].

You use bit masks and shifts for packing / unpacking. “But bitwise operators apply only to integer types in .Net, so you need to jump over some hoops!”

A good COBOL or C artist can point you in the right direction.

Find one of the old guys and pay your dues (about three beers should do this).

0
source share

All Articles