Nothing Special   »   [go: up one dir, main page]

WO2012083336A1 - Processing engine - Google Patents

Processing engine Download PDF

Info

Publication number
WO2012083336A1
WO2012083336A1 PCT/AU2010/001735 AU2010001735W WO2012083336A1 WO 2012083336 A1 WO2012083336 A1 WO 2012083336A1 AU 2010001735 W AU2010001735 W AU 2010001735W WO 2012083336 A1 WO2012083336 A1 WO 2012083336A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
unstructured
descriptor
processing engine
mappable
Prior art date
Application number
PCT/AU2010/001735
Other languages
French (fr)
Inventor
Michael Colin Berrington
Cameron Griffiths
Original Assignee
Financial Reporting Specialists Pty Limited Atf Frs Processes Trust
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Financial Reporting Specialists Pty Limited Atf Frs Processes Trust filed Critical Financial Reporting Specialists Pty Limited Atf Frs Processes Trust
Priority to PCT/AU2010/001735 priority Critical patent/WO2012083336A1/en
Publication of WO2012083336A1 publication Critical patent/WO2012083336A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2291User-Defined Types; Storage management thereof

Definitions

  • the present invention relates to the processing and egorisation of large volumes of data and, more particularly but not exclusively to financial reporting data.
  • the technical problem to be addressed is that of how to take a databit stream coded according to a first protocol and representing unstructured data and apply logical processing to that databit stream so as to produce tocol and representing structured data.
  • the structured a can then have further processing performed on it so as achieve outcomes that could not be achieved either at all or in a reasonable time frame but for the preceding logical processing step.
  • the processing engine of a preferred embodiment of the present invention when applied to a financial database automatically "tags” (also commonly referred to as “maps” or “allocates") information from an ined function codes, groups or fields; thus identifying populating .new and usable ways of filtering and regating the financial database information.
  • tags also commonly referred to as “maps” or “allocates”
  • a processing engine for processing unstructured data .located in an unstructured data table; said data comprising quantity data associated with an unstructured descriptor element; said engine comparing each said unstructured descriptor element against a master list of mappable data descriptors to create a single data description per entry; in the event of a match between an unstructured descriptor element and a mappable data descriptor in the mappable data descriptors list then quantity data associated with said unstructured data element is associated with the matching mappable data descriptor and copied into a partly structured data table.
  • a processing engine which receives a databit stream coded according to a first protocol and representing unstructured data; said processing engine applying logical ;abit stream coded according to a second protocol and >resenting structured data.
  • a processing engine which automatically "tags” (also commonly referred to as “maps” or “allocates”) information ' from an already existing financial database with, newly created and defined function codes, groups or fields; thus identifying and populating new and usable ways of filtering and aggregating the financial database information.
  • Preferably said partly structured data table is reviewed manually.
  • those unstructured descriptor elements for which no match is available are manually reviewed and a manual :ch made so as to produce a fully structured data table >m said partially structured data table.
  • those unstructured descriptor elements for which no match is available comprise less than 25 per cent of said unstructured data.
  • said unstructured data comprises accounting data.
  • the structured data table can be filtered and aggregated by all data fields, including newly created data descriptions, thereby to provide new ways of viewing, summarising and disseminating data.
  • a processing engine for processing unstructured data located in an unstructured data ' table in a computer memory; said data comprising quantity data associated with an unstructured descriptor element; said engine including a ment against a master list of mappable data descriptors create a single data description 1 per entry; in the event a match signal from said comparator indicating a match between an unstructured descriptor element and a mappable data descriptor in the mappable data descriptors list then quantity data associated with said unstructured data element is associated with, the matching mappable data descriptor and copied into a partly structured data table in said computer memory.
  • said database and said comparator are implemented via a microprocessor in communication with a computer memory.
  • Figure 1 is a block diagram of dataflow through a processing engine in accordance with a first preferred embodiment of the present invention
  • FIG. 2 is a logic flow diagram of processing steps applied to data in accordance with the first embodiment
  • Figure 3 illustrates data mapping for a particular instance of data
  • Figure 4 is an example of a fully structured data de
  • Fig 5 illustrates a further example of the tagging process where translation is to a common descriptor base - in this case an XBRL common descriptor set,
  • Fig 6 is a simplified explanation in table form of an embodiment of the technology process showing how the Master Chart Of Accounts tag may be determined.
  • the processing engine 10 comprises a translator 11 which receives input from comparator 12.
  • the comparator 12 receives and compares data in unstructured data table 13 and mappable data descriptor table 14.
  • the output of translator 11 is input to partly structured table 15.
  • a manual processing operation is then applied to table 15, the output of which results in a fully structured data table 16.
  • FIG. 2 a logic flow diagram is provided for processing steps applied to data passing tance, the data relates to trial balance data.
  • the processing engine is termed the itoallocator" in Figure 2.
  • Figure 3 illustrates data mapping in the instance where data is trial balance data and comprises a listing of unstructured descriptor elements 17 and a corresponding listing of mappable data descriptors 18. So, for example, unstructured descriptor element 17A comprising the •descriptor "till floats" maps to 'a matched mappable data descriptor 18a being descriptor "cash on hand.”
  • the processing engine automatically "tags” (also commonly referred to as “maps” or “allocates”) information ated and defined function codes, groups or fields; thus ntifying and populating new and usable ways of filtering aggregating the financial database information.
  • tags also commonly referred to as “maps” or “allocates”
  • the "tagging” (also commonly referred to as “mapping” or “allocating”) process uses elements of existing financial database fields (such as the general ledger code, account name / description or balance) to predict a new field (such as a Master Chart of Accounts category or XBRL Value) using, amongst other describable processes, a matrix and array linear pattern.
  • the technology also contains an inbuilt validation process, largely centred around the GL Code (with ranges input to match the current GL, ' which varies from system to system) and the GL Balance.
  • the tagging process is instantaneous - once you import the financial database (e.g. the trial balance), the new function codes (e.g. Master ' Chart of Accounts) appears immediately.
  • 5 illustrates a further example of the tagging process re translation is to a common descriptor base - in this e an XBRL common descriptor set.
  • Fig 6 is a simplified explanation of the technology process as to how it determines the Master Chart Of Accounts tag.
  • a fully structured data table 16 is illustrated with reference to a particular example.
  • the unstructured descriptor "Westpac account” is associated with quantity .ql . It is mapped by way of the table of Figure 3 to descriptor “cash at bank” associated with quantity ql.
  • the descriptor "trade debtors,” is initially associated with quantity q2 and is mapped to be associated with descriptor "trade receivables.”
  • mappable data descriptor table the smaller the amount of manual allocation will be as a proportion of total amount of data processed thereby resulting in the potential for relatively rapid processing of large volumes of data presenting as unstructured data from disparate sources.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

A processing engine which automatically "tags" (also commonly referred to as "maps" or "allocates") information from an already existing financial database with newly created and defined function codes, groups or fields; thus identifying and populating new and usable ways of filtering and aggregating the financial database information. The biggest advantages of this are elimination of time for manual creation; and also reliable and consistent treatment of these new populations thus significantly reducing errors.

Description

PROCESSING ENGINE
The present invention relates to the processing and egorisation of large volumes of data and, more particularly but not exclusively to financial reporting data.
BACKGROUND
Computers are known for their ability to process large volumes of data. However, in the case where the data, or at least parts of it, is unstructured there can be difficulties in the computer recognising the information contained in the data and knowing how to handle it.
There are many instances in industry where unstructured data presents a processing problem to data processing devices. For example accounting information in the form of trial balances from general ledger systems will present to a data processing system in an unstructured way and typically a different way depending on the general ledger system from which the data is derived.
The technical problem to be addressed is that of how to take a databit stream coded according to a first protocol and representing unstructured data and apply logical processing to that databit stream so as to produce tocol and representing structured data. The structured a can then have further processing performed on it so as achieve outcomes that could not be achieved either at all or in a reasonable time frame but for the preceding logical processing step.
It is an object of the present invention to address or at least ameliorate some of the above disadvantages.
Notes
1. The term "comprising" (and grammatical variations thereof) is used in this specification in the inclusive sense of "having" or "including", and not in the exclusive sense of "consisting only of".
2. The above discussion of the prior art in the .
Background of the invention, is not an admission that any information discussed therein is citable prior art or part of the common general knowledge of persons skilled in the art in any country. BRIEF DESCRIPTION OF INVENTION
In a broad form the processing engine of a preferred embodiment of the present invention when applied to a financial database automatically "tags" (also commonly referred to as "maps" or "allocates") information from an ined function codes, groups or fields; thus identifying populating .new and usable ways of filtering and regating the financial database information. The biggest advantages of this are elimination of time for manual creation; and also reliable .and consistent treatment of these new populations thus significantly reducing errors.
' Accordingly, in a further broad form of the present invention there is provided a processing engine for processing unstructured data .located in an unstructured data table; said data comprising quantity data associated with an unstructured descriptor element; said engine comparing each said unstructured descriptor element against a master list of mappable data descriptors to create a single data description per entry; in the event of a match between an unstructured descriptor element and a mappable data descriptor in the mappable data descriptors list then quantity data associated with said unstructured data element is associated with the matching mappable data descriptor and copied into a partly structured data table.
In a further broad form of the present invention there is provided a processing engine which receives a databit stream coded according to a first protocol and representing unstructured data; said processing engine applying logical ;abit stream coded according to a second protocol and >resenting structured data.
In a further broad form of the present invention there is provided a method for converting a databit stream coded according- to a first protocol and representing unstructured ■data via a step of logical processing to that databit stream so as to produce a second databit stream coded according to a second protocol and representing structured data.
In yet a further broad form of the present invention there is provided a processing engine which automatically "tags" (also commonly referred to as "maps" or "allocates") information' from an already existing financial database with, newly created and defined function codes, groups or fields; thus identifying and populating new and usable ways of filtering and aggregating the financial database information.
Preferably said partly structured data table is reviewed manually.
Preferably those unstructured descriptor elements for which no match is available are manually reviewed and a manual :ch made so as to produce a fully structured data table >m said partially structured data table.
Preferably those unstructured descriptor elements for which no match is available comprise less than 25 per cent of said unstructured data.
Preferably said unstructured data comprises accounting data. -
Preferably the structured data table can be filtered and aggregated by all data fields, including newly created data descriptions, thereby to provide new ways of viewing, summarising and disseminating data.
Preferably advantages of employing the processing engine are elimination of time for manual creation; and also reliable and consistent treatment of these new data descriptions thus significantly reducing errors.
In yet a further broad form of the invention there is provided a processing engine for processing unstructured data located in an unstructured data ' table in a computer memory; said data comprising quantity data associated with an unstructured descriptor element; said engine including a ment against a master list of mappable data descriptors create a single data description1 per entry; in the event a match signal from said comparator indicating a match between an unstructured descriptor element and a mappable data descriptor in the mappable data descriptors list then quantity data associated with said unstructured data element is associated with, the matching mappable data descriptor and copied into a partly structured data table in said computer memory.
'
Preferably said database and said comparator are implemented via a microprocessor in communication with a computer memory. BRIEF DESCRIPTION OF DRAWINGS
Embodiments of the present invention will now be described with reference to the accompanying drawings wherein:
Figure 1, . is a block diagram of dataflow through a processing engine in accordance with a first preferred embodiment of the present invention,
Figure 2, is a logic flow diagram of processing steps applied to data in accordance with the first embodiment,
Figure 3, illustrates data mapping for a particular instance of data, Figure 4 is an example of a fully structured data de,
Fig 5 illustrates a further example of the tagging process where translation is to a common descriptor base - in this case an XBRL common descriptor set,
Fig 6 is a simplified explanation in table form of an embodiment of the technology process showing how the Master Chart Of Accounts tag may be determined.
DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
With reference to Figure 1 there is illustrated a processing engine 10 in accordance with a first preferred embodiment of the present invention. In this instance the processing engine 10 comprises a translator 11 which receives input from comparator 12. The comparator 12 receives and compares data in unstructured data table 13 and mappable data descriptor table 14. The output of translator 11 is input to partly structured table 15. A manual processing operation is then applied to table 15, the output of which results in a fully structured data table 16.
With reference to Figure 2 a logic flow diagram is provided for processing steps applied to data passing tance, the data relates to trial balance data. In this tance the processing engine is termed the itoallocator" in Figure 2.
FRS AutoAllocator - outline
Using the FRS AutoAllocator (part of the FRS Process), we can define the process as:
' Importing Trial Balances from any General Ledger system and the FRS System (part of the FRS Process) automatically knows what Master Chart Of Account Allocation (based on data input from the International Financial Reporting Standards) to give it.
For example it would recognise:
GL Account Name Master Chart
Till floats as Cash on hand
estpac account as Cash at bank
Figure 3 illustrates data mapping in the instance where data is trial balance data and comprises a listing of unstructured descriptor elements 17 and a corresponding listing of mappable data descriptors 18. So, for example, unstructured descriptor element 17A comprising the •descriptor "till floats" maps to 'a matched mappable data descriptor 18a being descriptor "cash on hand."
In summary the processing engine automatically "tags" (also commonly referred to as "maps" or "allocates") information ated and defined function codes, groups or fields; thus ntifying and populating new and usable ways of filtering aggregating the financial database information. The biggest advantages of this are elimination of time for manual creation; and also reliable and consistent treatment of these new populations thus significan ly reducing errors.
Explanation of tagging process
The "tagging" (also commonly referred to as "mapping" or "allocating") process uses elements of existing financial database fields (such as the general ledger code, account name / description or balance) to predict a new field (such as a Master Chart of Accounts category or XBRL Value) using, amongst other describable processes, a matrix and array linear pattern. The technology also contains an inbuilt validation process, largely centred around the GL Code (with ranges input to match the current GL,' which varies from system to system) and the GL Balance.
The tagging process is instantaneous - once you import the financial database (e.g. the trial balance), the new function codes (e.g. Master' Chart of Accounts) appears immediately. 5 illustrates a further example of the tagging process re translation is to a common descriptor base - in this e an XBRL common descriptor set. Fig 6 is a simplified explanation of the technology process as to how it determines the Master Chart Of Accounts tag.
In Use
With reference to Figure 4 a fully structured data table 16 is illustrated with reference to a particular example. In this instance the unstructured descriptor "Westpac account" is associated with quantity .ql . It is mapped by way of the table of Figure 3 to descriptor "cash at bank" associated with quantity ql. Similarly, the descriptor "trade debtors," is initially associated with quantity q2 and is mapped to be associated with descriptor "trade receivables."
In the case of the descriptor "special advance" there is no mapping available from the table of Figure 3 so, initially, no mapping takes place in the creation of the partially structured data table. The manual data entry step is then performed whereby the descriptor "special advance" is mapped to "other receivables" and remains associated with quantity qn.
Having produced the fully structured data table the it in an automated fashion, it will be appreciated that more comprehensive the mappable data descriptor table the smaller the amount of manual allocation will be as a proportion of total amount of data processed thereby resulting in the potential for relatively rapid processing of large volumes of data presenting as unstructured data from disparate sources.
The above describes only some embodiments of the present invention and modifications, obvious to those skilled in the art, can be made thereto without departing from the scope of the present invention.

Claims

A processing engine for processing unstructured data located in an unstructured data table; said data comprising quantity data associated with an unstructured descriptor element; said engine comparing each said unstructured descriptor element against a master list of mappable data descriptors to create a single data description per entry; in the event of a match between an unstructured descriptor element and a mappable data descriptor in the mappable data descriptors list then quantity data associated with said unstructured data element is associated with the matching mappable data descriptor and copied into a partly structpred data table.
The processing engine of claim 1 wherein said partly structured data table is reviewed manually
The processing engine of claim 2 wherein those unstructured descriptor elements for which no match is available are manually reviewed and a manual match made so as to produce a fully structured data table from said partially structured data table. The processing engine of claim 3 wherein those unstructured descriptor elements for which no match is available comprise less than 25 per cent of said
5. The processing engine ' of claim 3 wherein said unstructured data comprises accounting data.
6. The processing engine of any previous claim wherein the structured data table can be filtered and aggregated by all data fields,, including newly created data descriptions, thereby to provide new ways of viewing, summarising and disseminating data.
7. The processing engine of any previous claim wherein advantages of employing the processing engine are elimination of time for manual creation; and also reliable and consistent treatment of these new data descriptions thus significantly reducing errors.
8. A processing engine for processing unstructured data located in an unstructured data table in a computer memory; said data comprising quantity data associated with an unstructured descriptor element; said engine including a comparator which compares each said unstructured descriptor element against a master list of mappable data descriptors to create a single data description per entry; in the event of a match signal from said comparator indicating a match between an unstructured descriptor element and a mappable data descriptor in the mappable data descriptors list then quantity data associated with said unstructured data element is associated with into a partly structured data table in said computer memory.
9. The processing engine of any previous claim wherein said database and said comparator are implemented via a microprocessor in communication with a computer memory.
10. A machine readable medium including executable code executable on a processor which implements the processing engine of any one of claims 1 to 9.
11. A processing engine which receives a databit stream coded according to a first protocol and representing unstructured data? said processing engine applying logical processing to that databit stream so as to produce a second databit stream coded according to a second protocol and representing structured data.
12. The processing engine of claim 11 wherein said databit stream comprises quantity data associated with an unstructured descriptor element; said engine including a comparator which compares each said unstructured descriptor element against a master list of mappable data descriptors to create a single data description per entry; in the event of a match between an unstructured descriptor element and a mappable data descriptor in the mappable data descriptors list then quantity data associated with said unstructured data element is associated with the matching mappable data descriptor and copied via said second databit stream into a partly structured data table in said computer memory.
The processing engine of claim 12 wherein said database and said comparator are implemented via a microprocessor in communication with a ■ computer memory.
A method for converting a databit stream coded according to a first protocol and representing unstructured data via a step of logical processing to that databit stream so as to . produce a second databit stream coded according to a second protocol and representing structured data.
The method of claim 14 wherein said step of logical processing is performed by a processing engine for processing unstructured data located in an unstructured data table in a computer memory; said unstructured data comprising quantity data associated with an unstructured descriptor element; said engine including a comparator which compares master list of mappable data descriptors to create a single data description per entry; in the event of a match signal from said comparator indicating a match between an unstructured descriptor element and a mappable data descriptor in the mappable data descriptors list then quantity data associated with said unstructured data element is associated with the matching mappable data descriptor and copied into a partly structured data table in said computer memory via said second databit stream coded according to a second protocol and representing structured data.
16. The method of claim 14 or 15 wherein said database and said comparator are implemented via a microprocessor in communication with a computer memory.
.. A processing engine which automatically "tags" (also commonly referred to as "maps" or "allocates") information from an already., existing financial database with newly created and defined function codes, groups or fields; thus identifying and populating new and usable ways of filtering and aggregating the financial database information.
PCT/AU2010/001735 2010-12-23 2010-12-23 Processing engine WO2012083336A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/AU2010/001735 WO2012083336A1 (en) 2010-12-23 2010-12-23 Processing engine

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/AU2010/001735 WO2012083336A1 (en) 2010-12-23 2010-12-23 Processing engine

Publications (1)

Publication Number Publication Date
WO2012083336A1 true WO2012083336A1 (en) 2012-06-28

Family

ID=46312874

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/AU2010/001735 WO2012083336A1 (en) 2010-12-23 2010-12-23 Processing engine

Country Status (1)

Country Link
WO (1) WO2012083336A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2003058504A1 (en) * 2001-12-27 2003-07-17 Fair Isaac Corporation Augmenting data in a database for predictive modeling
US20050108256A1 (en) * 2002-12-06 2005-05-19 Attensity Corporation Visualization of integrated structured and unstructured data
US7668849B1 (en) * 2005-12-09 2010-02-23 BMMSoft, Inc. Method and system for processing structured data and unstructured data

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2003058504A1 (en) * 2001-12-27 2003-07-17 Fair Isaac Corporation Augmenting data in a database for predictive modeling
US20050108256A1 (en) * 2002-12-06 2005-05-19 Attensity Corporation Visualization of integrated structured and unstructured data
US7668849B1 (en) * 2005-12-09 2010-02-23 BMMSoft, Inc. Method and system for processing structured data and unstructured data

Similar Documents

Publication Publication Date Title
CN110383319B (en) Large scale heterogeneous data ingestion and user resolution
CN111475513B (en) Form generation method and device, electronic equipment and medium
JP5087261B2 (en) Data element naming system and method
CN110083623B (en) Business rule generation method and device
CN108268593B (en) Method, device, server and storage medium for processing credit card insurance information
CN112541745A (en) User behavior data analysis method and device, electronic equipment and readable storage medium
CN103428080A (en) Data unpacking and data packing method
CN109086433A (en) A kind of file management method and server based on big data analysis
CN113434674A (en) Data analysis method and device, electronic equipment and readable storage medium
CN112734543A (en) Accounting processing method and device, computer equipment and storage medium
CN111985194A (en) Data storage method and device, electronic equipment and storage medium
CN105389346A (en) Bidding clearing method and apparatus in electronic bidding evaluation system
CN111159183A (en) Report generation method, electronic device and computer readable storage medium
CN113434542B (en) Data relationship identification method and device, electronic equipment and storage medium
CN110097444A (en) A kind of flowing water code generating method and device
CN112541688A (en) Service data checking method and device, electronic equipment and computer storage medium
CN112101889A (en) Metadata-based business flow control method and device
AU2009233605B2 (en) Processing Engine
WO2012083336A1 (en) Processing engine
US20150213098A1 (en) Business Rules Influenced Quasi-Cubes with Higher Diligence of Data Optimization
CN101425167A (en) Method for generating and parsing trading information
CN110648219B (en) Method and device for standardizing input area of bank transaction system
CN106570004B (en) Data management method and device
CN115034674A (en) Charging management method, system and electronic equipment
CN103092549A (en) Representing method and device for fund change situation

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 10861183

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 10861183

Country of ref document: EP

Kind code of ref document: A1