Alpha Software Mobile Development Tools:   Alpha Anywhere    |   Alpha TransForm subscribe to our YouTube Channel  Follow Us on LinkedIn  Follow Us on Twitter  Follow Us on Facebook



The Alpha Software Forum Participation Guidelines

The Alpha Software Forum is a free forum created for Alpha Software Developer Community to ask for help, exchange ideas, and share solutions. Alpha Software strives to create an environment where all members of the community can feel safe to participate. In order to ensure the Alpha Software Forum is a place where all feel welcome, forum participants are expected to behave as follows:
  • Be professional in your conduct
  • Be kind to others
  • Be constructive when giving feedback
  • Be open to new ideas and suggestions
  • Stay on topic

Be sure all comments and threads you post are respectful. Posts that contain any of the following content will be considered a violation of your agreement as a member of the Alpha Software Forum Community and will be moderated:
  • Spam.
  • Vulgar language.
  • Quotes from private conversations without permission, including pricing and other sales related discussions.
  • Personal attacks, insults, or subtle put-downs.
  • Harassment, bullying, threatening, mocking, shaming, or deriding anyone.
  • Sexist, racist, homophobic, transphobic, ableist, or otherwise discriminatory jokes and language.
  • Sexually explicit or violent material, links, or language.
  • Pirated, hacked, or copyright-infringing material.
  • Encouraging of others to engage in the above behaviors.

If a thread or post is found to contain any of the content outlined above, a moderator may choose to take one of the following actions:
  • Remove the Post or Thread - the content is removed from the forum.
  • Place the User in Moderation - all posts and new threads must be approved by a moderator before they are posted.
  • Temporarily Ban the User - user is banned from forum for a period of time.
  • Permanently Ban the User - user is permanently banned from the forum.

Moderators may also rename posts and threads if they are too generic or do not property reflect the content.

Moderators may move threads if they have been posted in the incorrect forum.

Threads/Posts questioning specific moderator decisions or actions (such as "why was a user banned?") are not allowed and will be removed.

The owners of Alpha Software Corporation (Forum Owner) reserve the right to remove, edit, move, or close any thread for any reason; or ban any forum member without notice, reason, or explanation.

Community members are encouraged to click the "Report Post" icon in the lower left of a given post if they feel the post is in violation of the rules. This will alert the Moderators to take a look.

Alpha Software Corporation may amend the guidelines from time to time and may also vary the procedures it sets out where appropriate in a particular case. Your agreement to comply with the guidelines will be deemed agreement to any changes to it.

Bonus TIPS for Successful Posting

Try a Search First
It is highly recommended that a Search be done on your topic before posting, as many questions have been answered in prior posts. As with any search engine, the shorter the search term, the more "hits" will be returned, but the more specific the search term is, the greater the relevance of those "hits". Searching for "table" might well return every message on the board while "tablesum" would greatly restrict the number of messages returned.

When you do post
First, make sure you are posting your question in the correct forum. For example, if you post an issue regarding Desktop applications on the Mobile & Browser Applications board , not only will your question not be seen by the appropriate audience, it may also be removed or relocated.

The more detail you provide about your problem or question, the more likely someone is to understand your request and be able to help. A sample database with a minimum of records (and its support files, zipped together) will make it much easier to diagnose issues with your application. Screen shots of error messages are especially helpful.

When explaining how to reproduce your problem, please be as detailed as possible. Describe every step, click-by-click and keypress-by-keypress. Otherwise when others try to duplicate your problem, they may do something slightly different and end up with different results.

A note about attachments
You may only attach one file to each message. Attachment file size is limited to 2MB. If you need to include several files, you may do so by zipping them into a single archive.

If you forgot to attach your files to your post, please do NOT create a new thread. Instead, reply to your original message and attach the file there.

When attaching screen shots, it is best to attach an image file (.BMP, .JPG, .GIF, .PNG, etc.) or a zip file of several images, as opposed to a Word document containing the screen shots. Because Word documents are prone to viruses, many message board users will not open your Word file, therefore limiting their ability to help you.

Similarly, if you are uploading a zipped archive, you should simply create a .ZIP file and not a self-extracting .EXE as many users will not run your EXE file.
See more
See less

CSV export escapement problem

  • Filter
  • Time
  • Show
Clear All
new posts

  • CSV export escapement problem

    I need to do a CSV export to MySQL from DBF. The DBF memo fields contain anything you can imagine. I first created a CSV separated by ^ to avoid most character problems, but there were issues. Now I am trying to do a more traditional CSV.

    If a text field contains a comma or quote, Alpha's export operation does not automatically escape them. I don't want to quote all character fields for other reasons. I am guessing "properties -> add filter rule" is supposed to be used, but I can't figure it out.

    field1 = 123
    field2 = my "quoted" text, with comma
    field3 = plain text
    desired csv = 123,"my ""quoted"" text, with comma",plain text

    And this is the Alpha interface.

    If I use an expression s_quote(","), then the entire field is replaced by ",", when what I want is the field to be quoted. And it could be any field.

    Anyone used this?

    Attached Files

  • #2
    You do not say if the memo field is a field in the table or the memo field is in a stand alone child table linked to a parent table as Dr Wayne recommends. Just wondering if the later would help solve you problem.
    Alpha 5 Version 11
    AA Build 2999, Build 4269, Current Build
    DBF's and MySql
    Desktop, Web on the Desktop and WEB

    Ron Anusiewicz


    • #3
      The memo field is in a separate table, but I am exporting from a mapped table that includes both - so it looks to Alpha like a single table.


      • #4
        I can export quotes and commas to a csv if I use a semi-colon as the delimiter. No need for any filters. Result using your sample would be:
        123;my "quoted" text, with comma;plain text
        If you add a filter for quotes (") with the expression
        the output is:
        123;'my "quoted" text, with comma';plain text
        Neither of these plays nicely with newer versions of Excel as Microsoft insist the default separator is ',' for a csv file. LibreOffice Calc always prompts for values for delimiter when you try to open a csv which makes troubleshooting a lot simpler.

        You don't mention what method you're using for the import to MySQL but it seems quite flexible as to what separators can be used.


        • #5
          I made some progress, but not there yet. This is the xbasic view of an export operation with character filters.
          a_tbl = table.current()
          export.type = 0
          export.names = .T.
          export.file = "csvtest6.asc"
          export.options = "LTS"
          export.field_sep = ","
          export.record_sep = "<CR><LF>"
          export.string_filter_rules = <<%ftrul%
          This code changes carriage returns to {CR}{LF} which I can convert back to carriage return after sql import. If a field has an embedded comma, the field is escaped, and that is fine​. Only if a field is escaped, any CRLF is converted instead to chr(13) +chr(10), which is only a problem if it is at the end of the field. In that case the import gets really screwed up. This occurs frequently and is not something that can be changed on the source side.

          This is a test file.
          abc,bill,"simple memo, but it does have comma and ending CR"+chr(13)+chr(10),15
          def,pipe | test,this has \ and cr{CR}{LF}to create second line.,
          jkl,"comma, and double \"quote\"","various \"quoted\" specials and comma, ",6
          The chr(13)+chr(10) in first line is what will cause a problem on import to MySQL, since it is at the end of the field. All would be OK if it came out as {CR}{LF}.



          • #6
            The road has continued to be winding. I had to ditch comma separated fields. Instead using ^ due to data issues. Below is the xbasic version of the operation, and the sequence of filter statements DOES make a difference. The remaining problem is that memo fields are truncated after export of 1024 characters. Not sure what is up with that?

            a_tbl = table.current()
            query.order = "Personid"
            query.filter = ".t."
            query.options = "I"
            export.type = 0
            export.names = .T.
            export.file = "sqlmaria_person.csv"
            export.options = "LTSN"
            export.field_sep = "^"
            export.record_sep = "<CR><LF>"
            export.string_filter_rules = <<%ftrul%


            • #7
              Hi Bill,
              I have experienced these issues many times. I will give you my solution that has worked over the years. I don't try and export the data and fix the data in one event. I have used an intermediary temporary table to export the data, and fix the data in the temp table before moving the data to the final resting place. It sometimes requires an added field in the temp table to store the fixed data and the fixed data field is what is used moving forward. Just a thought for you.
              Mike W
              "I rebel in at least small things to express to the world that I have not completely surrendered"


              • #8
                Thanks Mike. What worked for me was a mapped table in which I do all my conversions. For example, for date fields I define the mapped field as character and then set the value as chr(27) if the value is {}, otherwise yyyy-MM-dd.. The export converts the chr(27) to \N. The other lesson was to never specify an Enclosed By character for this particular set of horrible data.