Geek Answers Handbook

File processing development that handles many file formats, parsing, checking and saving

If you needed to develop a component / file processing system that can accept a wide variety of file formats (including proprietary formats such as Excel), analyze / check and store this information in the database .. How would you do this?

NOTE: 95% of the time, 1 line of input will be equal to one record in the database, but not always.

Currently, I am using some special software that I have developed to analyze / verify / store client data in our database. The system identifies the file by its location in the file system (from ftp crashes), and then loads the XML definition file. (The correct XML is loaded based on where the input file was downloaded).

XML defines things like file layout (delimited or fixed width) and field-specific elements (length, data type (numeric, alpha, alphanumeric) and which DB column to store the field).

         <delimiter><![CDATA[ ]]></delimiter>
   <numberOfItems>12</numberOfItems>
   <dataItems>
    <item>
     <name>Member ID</name>
     <type>any</type>
     <minLength>0</minLength>
     <maxLength>0</maxLength>
     <validate>false</validate>
     <customValidation/>
     <dbColumn>MembershipID</dbColumn>
    </item>

Because of this design, the input files must be text (fixed width or delimiter) and have a 1 to 1 relationship from the data field of the input files in the DB column.

I would like to expand the capabilities of our file processing system to take Excel or other file formats.

, , , , .

: , ( , Excel), / .. ?

+5
3

, , , - LinkedIn, LinkedIn, .

0

, - - ...

+-----------+
| reader1   |
|           |---
+-----------+   \---
                    \---   +----------------+               +-------------+
                        \--|  validation    |               |  DB         |
                       /---|                |---------------|             |
+-----------+    /-----    +----------------+               +-------------+
| reader2   |----
|           |
+-----------+

( ?) , Validation -, DB... .

, , , - Generic ReaderToValidator. -. , , G.R.2.V. .

, GR2V Validator , GR2V ReadParseFile. , . , .

+1

SO , , . . .

Although such discussions should be technologically agnostic, I suspect that you will probably find that the Java and .Net camps do not meet too much. I would look at "Server Side", but I am doing Java and therefore looking for Java material.

0
source

More articles:


All Articles