Alpha Software Mobile Development Tools:   Alpha Anywhere    |   Alpha TransForm subscribe to our YouTube Channel  Follow Us on LinkedIn  Follow Us on Twitter  Follow Us on Facebook

Announcement

Collapse

The Alpha Software Forum Participation Guidelines

The Alpha Software Forum is a free forum created for Alpha Software Developer Community to ask for help, exchange ideas, and share solutions. Alpha Software strives to create an environment where all members of the community can feel safe to participate. In order to ensure the Alpha Software Forum is a place where all feel welcome, forum participants are expected to behave as follows:
  • Be professional in your conduct
  • Be kind to others
  • Be constructive when giving feedback
  • Be open to new ideas and suggestions
  • Stay on topic


Be sure all comments and threads you post are respectful. Posts that contain any of the following content will be considered a violation of your agreement as a member of the Alpha Software Forum Community and will be moderated:
  • Spam.
  • Vulgar language.
  • Quotes from private conversations without permission, including pricing and other sales related discussions.
  • Personal attacks, insults, or subtle put-downs.
  • Harassment, bullying, threatening, mocking, shaming, or deriding anyone.
  • Sexist, racist, homophobic, transphobic, ableist, or otherwise discriminatory jokes and language.
  • Sexually explicit or violent material, links, or language.
  • Pirated, hacked, or copyright-infringing material.
  • Encouraging of others to engage in the above behaviors.


If a thread or post is found to contain any of the content outlined above, a moderator may choose to take one of the following actions:
  • Remove the Post or Thread - the content is removed from the forum.
  • Place the User in Moderation - all posts and new threads must be approved by a moderator before they are posted.
  • Temporarily Ban the User - user is banned from forum for a period of time.
  • Permanently Ban the User - user is permanently banned from the forum.


Moderators may also rename posts and threads if they are too generic or do not property reflect the content.

Moderators may move threads if they have been posted in the incorrect forum.

Threads/Posts questioning specific moderator decisions or actions (such as "why was a user banned?") are not allowed and will be removed.

The owners of Alpha Software Corporation (Forum Owner) reserve the right to remove, edit, move, or close any thread for any reason; or ban any forum member without notice, reason, or explanation.

Community members are encouraged to click the "Report Post" icon in the lower left of a given post if they feel the post is in violation of the rules. This will alert the Moderators to take a look.

Alpha Software Corporation may amend the guidelines from time to time and may also vary the procedures it sets out where appropriate in a particular case. Your agreement to comply with the guidelines will be deemed agreement to any changes to it.



Bonus TIPS for Successful Posting

Try a Search First
It is highly recommended that a Search be done on your topic before posting, as many questions have been answered in prior posts. As with any search engine, the shorter the search term, the more "hits" will be returned, but the more specific the search term is, the greater the relevance of those "hits". Searching for "table" might well return every message on the board while "tablesum" would greatly restrict the number of messages returned.

When you do post
First, make sure you are posting your question in the correct forum. For example, if you post an issue regarding Desktop applications on the Mobile & Browser Applications board , not only will your question not be seen by the appropriate audience, it may also be removed or relocated.

The more detail you provide about your problem or question, the more likely someone is to understand your request and be able to help. A sample database with a minimum of records (and its support files, zipped together) will make it much easier to diagnose issues with your application. Screen shots of error messages are especially helpful.

When explaining how to reproduce your problem, please be as detailed as possible. Describe every step, click-by-click and keypress-by-keypress. Otherwise when others try to duplicate your problem, they may do something slightly different and end up with different results.

A note about attachments
You may only attach one file to each message. Attachment file size is limited to 2MB. If you need to include several files, you may do so by zipping them into a single archive.

If you forgot to attach your files to your post, please do NOT create a new thread. Instead, reply to your original message and attach the file there.

When attaching screen shots, it is best to attach an image file (.BMP, .JPG, .GIF, .PNG, etc.) or a zip file of several images, as opposed to a Word document containing the screen shots. Because Word documents are prone to viruses, many message board users will not open your Word file, therefore limiting their ability to help you.

Similarly, if you are uploading a zipped archive, you should simply create a .ZIP file and not a self-extracting .EXE as many users will not run your EXE file.
See more
See less

Screen Scraping Data Capture...??

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Screen Scraping Data Capture...??

    Hi all you Good People,
    I am currently developing an application for a Gas Pipeline Contractor who is doing connections to homes, offices etc.. ..development in Alpha5V4 of course. The sub-contractor uses the Gas Company system to log in via the Internet (Extranet)and get their jobs notified to them etc... however they (the GASCO) have not made available any facility to download data via FTP to their sub-contractors. The only option available is to screen-print the data and then rekey... we are not happy with that since they will be doing 20-30 jobs a day with multiple crews.

    My problem therefore is ... does anyone have some ideas on how I might capture data from a display only screen ...using X-Basic I think .. or are you aware of any other tools which would do this ????

    I have looked longingly at the DDE section and the Clipboard secton in Alpha5 / X-basic manuals but neither seem to offer a solution.....

    Any thoughts, wisdom, insights out there... I have not yet seen a problem on the board which has defeated this community ! Thank you for your help previously and in anticipation of further wisdom.
    Best Regards
    Tom





    #2
    RE: Screen Scraping Data Capture...??

    There are some utilities which will "read" a webpage and output to different file types (ASCII, etc), but can't think of any names/sources off hand. I've heard that you can also do this in Java or XML. You could always have your client highlight and copy/paste the data to A5 to avoid rekeying.

    Comment


      #3
      RE: Screen Scraping Data Capture...??

      You could have them fax it to you, or fax it to yourself, then there are programs that will convert the fax to data entry. Cann't think of any names right off, but they advertise in the back of PC magazines.

      Good Luck

      Tom

      Comment


        #4
        RE: Screen Scraping Data Capture...??

        Hi Tom,
        Seems like I'm always running into this type of problem, getting different systems to talk to each other. How is the data presented to you? All jobs lumped together in a single html file or is it a seperate html for each job?
        If it's all lumped together you could probably capture and write it to an ascii file and write an A5 script to parse through the ascii file. Not a perfect soluction but sure would be better than re-keying.

        Good luck,
        Jim

        Comment


          #5
          RE: Screen Scraping Data Capture...??

          thanks for your collective help.
          the screens come one per job, with all key data on a screen inquiry. I will continue to pursue it and will advise outcome if it is of general interest.
          thanks & regards
          tom

          Comment


            #6
            RE: Screen Scraping Data Capture...??

            Hi Again,

            I just got and read me PC Magazine and they reviewed the new AskSam program that will take free text and convert it to a database. The article did not say what it it would export to, but I imagine it will export to something you could use with A5. askSam 4.0 list price $149.95.

            Hope no one takes offense

            Tom

            Comment


              #7
              RE: Screen Scraping Data Capture...??

              Tom:

              Does the notification screen to which you log on have a constant format. If so, you should be able to take it as text and parse it. If it is just free text in no particular order I don't see how you could distinguish a telephone number from a date.

              Bill
              Bill Hanigsberg

              Comment


                #8
                RE: Screen Scraping Data Capture...??

                I remember a product alpha used to carry called report-styler whose sole purpose was converting reports back into data without programming (i.e. the user 'painted' the areas in a preview window and would say 'this is a name field' 'this is a date' etc -- and once one was defined, you could use the same template over and over again to import data of the same format.

                I don't know if this product alpha licensed is available any more (I doubt it) - but I will find if anyone is still selling it -- I always thought it was a cool idea for a product, and invaluable if you needed it.

                Can you post a couple samples of the data? if the format is regular enough (I.e. fields are always at same place on screen) - it may be a trivial xbasic program to write if everything falls in 'one' place.

                Comment


                  #9
                  RE: Screen Scraping Data Capture...??

                  Tom,

                  Monarch by Datawatch does the same thing to a report printed as a text file. You define the data area and set up a map to convert the data. The maps are called models. From a command line you can read data, convert it to fields and export it to another file. It really is designed to extract lots of data from large reports, but it might work for you if you have a consistent output to a text file.

                  John

                  Comment


                    #10
                    RE: Screen Scraping Data Capture...??

                    Hi Cian,

                    >>it may be a trivial xbasic program to write if everything falls in 'one' place

                    Comment


                      #11
                      RE: Screen Scraping Data Capture...??

                      Thanks all for your help. The mainframe data is structured consistently in a screeen layout...(as say A5 would be)...same fields in same place etc.. The Monarch idea has given me a clue and may work..screen print the image and they look to convert the captured data to a record.

                      Also I recall from a long time ago being able to specifically address the 1920-byte 'screen-buffer' in Cobol !(sic) and this could work in reverse. The concept is as you say Jim in that the 'other application' screen is 'read' and then reformatted to provide data for another application.....

                      It is the most basic or data transfer!.... I was thinking if I could address the screen-buffer, I could write an X-basic to capture the data.
                      More R&D... many thanks everyone.
                      I will advise progress since it is of some interest it seems !
                      Best Regards
                      tom.

                      Comment


                        #12
                        RE: Screen Scraping Data Capture...??

                        thanks for the followup Cian. The screen data is totally regular... everything in the same place on each job screen.

                        You are quite right about it being a cool idea as it is the basis of a COLD application (no pun intended)- computer output to laser disc.... this is where the electronic data from Invoices, statements, documents etc is compressed into one large record with field numbers and can then be reconstituted into templates for viewing by merely placing the field number on the appropriate box in the template for screen display ..... the "Filenet" system uses the concept extensively for e-document archiving...!
                        Thanks again for the sssistance.... I will copy a mainframe screen and attach it later.....
                        Best Regards
                        tom

                        Comment


                          #13
                          RE: Screen Scraping Data Capture...??

                          Initially I was thinking of just processing the text file. I would
                          have to investigate the best way of doing live 'terminal' screen reading.

                          Presumably Win32 code can access a DOS sessions screen data because of the
                          fact that Cut/Copy/Paste commands which appear on a console windows system
                          menu and trasfer to/from the windows clipboard. My first look for such a call
                          in the Win32 API is discouraging - I cannot find a way to get access to a
                          Console handle that I do not own (maybe A5 could spawn the task - the problem there is that we don't get control back untill that task has completed).

                          This would be an interesting project - I would like to see A5 get better
                          at aquiring data from any source (including a telnet or command window).

                          I have used EMACS (under linux) on occasion and am very impressed by the
                          ability of EMACS to spawn a command shell environment *inside* the editor -
                          meaning that you can use all the editor capabilities to cut&paste from a live
                          shell session -- maybe the approach could be the same in A5 if I trap standard
                          input/output from a dos application and redirect it to alpha fives interactive
                          window (no promises - I don't know how much work that would be yet). The opensources site for debian linux showed some promise - it appears to open & manipluate a console app by redirecting the consoles IO - its a pretty prodigious
                          amount of code though...


                          Also - this may be yet another reason for writing a Win32 Command line interface
                          to xbasic - maybe it would be easier to aquire data from a dos window if xbasic
                          was running inside the dos window.

                          Another alternative I could try - using a telnet server for windows, and run
                          the dos apps in a telnet session - and create a telnet client that passes on
                          the information to alpha five -- this would allow us to aquire data from any
                          source that supports telnet interface (i.e. mainframes) -- writing a telnet
                          client would be a bit of work, but perhaps we could buy & modify one.

                          Comment


                            #14
                            RE: Screen Scraping Data Capture...??

                            Hi Cian

                            >>Also - this may be yet another reason for writing a Win32 Command line interface to xbasic ....

                            Another alternative I could try - using a telnet server for windows, and run the dos apps in a telnet session - and create a telnet client that passes on the information to alpha five

                            Comment


                              #15
                              RE: Screen Scraping Data Capture...??

                              This proved to interesting a problem to pass up if I could manage to figure out
                              how to allocate a console screen in A5 and use it for other apps --
                              In version 5.0 I just added a new function:

                              sys_shell_read() - which returns the contents of a A5's console window (if
                              any exists) --

                              The rest is done with declares

                              ' a5 version 5.0 code to capture console mode screen input
                              declare kernel32.dll AllocConsole L
                              declare kernel32.dll FreeConsole L

                              AllocConsole()

                              sys_shell("cmd")
                              ...
                              use sys_shell_read() whenever you want to read the screen

                              ... User needs to take control of the console window - start application
                              etc...

                              My test application allowed me to use standard DOS apps in the command window
                              that A5 launched, and read the latest screen into A5 at 5 second intervals...

                              I included a screen shot for those curious...

                              Comment

                              Working...
                              X