GL transaction data extract

We extract GL transaction data from Sage 100 Advanced (Providex db) for the prior month once the month has been closed. These transactions are then imported into NetSuite for consolidation - we have multiple companies running different accounting systems and they all go through this process every month - Sage 100 is just one piece of the puzzle.

The problem is that it takes a very long time to extract the data from the GL_DetailPosting table using a date range - so long that sometimes it fails and we have to retry several times before we have success.

I have attached the GL_DetailPosting table to an Access database and run a simple query to extract just the month's transactions into a new Access table, then export the data from the new table. It should be simple enough.

Is there any easier/better way of doing this?

We want to avoid the frustration of running and failing all the time.

Parents
  • 0
    Hi, the PostingDate is a secondary key in the GL_DetailPosting table. If your query is failing I would suspect that your query is not optimized. If you'll post your query and the total number of records in GL_DetailPosting I'll take a look.
    John Nichols
    Sage
  • 0 in reply to jcnichols
    2 million records in GL_DetailPosting, November returns about 45000 records.
    Query is:
    SELECT SourceJournal, JournalRegisterNo, DocumentNo, AccountKey, PostingDate, SequenceNo, PostingComment, DebitAmount, CreditAmount
    INTO Detail
    FROM GL_DetailPosting
    WHERE PostingDate >=#11/1/2016# and PostingDate <=#11/30/2016#

    Note that GL_DetailPosting is linked in an Access db and Detail is created by the query.
Reply
  • 0 in reply to jcnichols
    2 million records in GL_DetailPosting, November returns about 45000 records.
    Query is:
    SELECT SourceJournal, JournalRegisterNo, DocumentNo, AccountKey, PostingDate, SequenceNo, PostingComment, DebitAmount, CreditAmount
    INTO Detail
    FROM GL_DetailPosting
    WHERE PostingDate >=#11/1/2016# and PostingDate <=#11/30/2016#

    Note that GL_DetailPosting is linked in an Access db and Detail is created by the query.
Children
  • 0 in reply to 49153
    I would strongly suggest switching from using a Linked table to a SQL Pass Through Query. It returns the data twice as fast.
  • 0 in reply to BigLouie
    Working on that right now, I have the test query running.
    I need to change my VB code but essentially I'm almost done.
  • 0 in reply to 49153
    Question about maintenance: I see Rebuild Sort Files and Rebuild Key Files under Utilities.
    Will any of these have any affect on query speed?
  • 0 in reply to BigLouie
    I ran the same query against a linked table and against a pass thru query - both run for the same time.
    I tried with CS ODBC on and off, it does not seem to make a difference.
  • 0 in reply to 49153
    Hi, Pulling records from a "single table" via a link table in Access it will be slower than going directly against the Providex DB. Pulling records from a smaller DB 180k records returns a months worth of records 5k in under 3 seconds. I'll work on creating the larger DB and follow up. Your query looks fine except that your not linking back to get the G/L Account Number. Thx John
  • 0 in reply to jcnichols
    I have a small company that exports in seconds.
    I extract the GL accounts in a separate query, then in Access I link the tables. It seems faster that way.
  • 0 in reply to 49153
    After some further testing I found some interesting results.
    My initial approach was to link the GL table to an Access db and use a "SELECT INTO" query to copy the required month's data into an Access table. This process took about 30 minutes to run - surprisingly slow.

    I changed to opening a recordset and then looping through the recordset and copying each record to an Access table:

    Do while not rs.EOF
    rsAccess.AddNew
    rsAccess("Field")=rs("Field").... copy required fields
    rsAccess.Update
    rs.MoveNext
    Loop

    I tested 3 recordsets:
    Access linked table
    Access passthru query
    Providex query using SOTAMAS90 DSN

    The results for copying 45,000 records are:
    Access query: 1:43 (that's 1 minute 43 seconds)
    Passthru query: 4:16
    Providex query: 1:56

    That is significantly faster than the SQL SELECT INTO query.
    An Access linked table query is the winner.