Advice on creating PDF reports containing lots of images

[pcsystems]

Hello. In the past, I've used the Alpha report generator to create columnar text/numeric reports in web apps with no problem. In my current web app, I need to produce PDF files containing catalogs of images (mostly JPGs) and their corresponding descriptions. It's not a complicated report, but it can have anywhere from a few images to several hundred, depending on the user.

The problem is that the freeform report doesn't orient/rotate the images properly when I generate it as a PDF. It simply uses a default orientation, without regard to how the user may have rotated their mobile device when they captured and uploaded the image. Also, I can't see any way for the PDF generator to properly decode supplementary characters for text stored as utf8mb4 (4-bytes/character instead of 3). Other than that, the report is formatted correctly on the page and contains the desired footers and page breaks.

If I generate the same report in HTML format, the images are all oriented properly and utf8mb4 characters decode correctly. However, there are no footers or page breaks in the output. In addition, generating a freeform PDF report with a lot of images pretty much takes forever, while the HTML rendering is much quicker. Either way, if there are lots of images to print, I can end up with a 413 error (payload too large) from the server.

Until now, I was under the impression if I generated an Alpha report as a PDF or as an HTML file, it would look the same and not take an eternity to create. Apparently that is not the case. So I am looking for a more efficient solution to producing PDF files for this application. Because of the orientation issue, it looks like generating as HTML is the only viable alternative in Alpha. And maybe using the "chunked responses" mode and the HTML_TO_PDF function is the answer, but if it is, I still don't know how to embed footers and page breaks in the HTML code at the appropriate spots.

Would using Alpha's built-in report generator still be your recommended approach in this case or is there a better/easier way? Am I missing some key point here or trying to implement this using the wrong coding for the report? Am looking for some direction or a resource to help me understand how to create a properly-formatted and printable PDF of any size, with images oriented correctly and text properly decoded.

A few months ago, I submitted a bug report with a test case to Alpha and ended up without any helpful response. I am using the latest Alpha stable release (8867) and publishing to Alpha Cloud. Two sample output files for the same "PrintContent" report are attached, one rendered as a PDF and one as HTML (saved as a PDF). In addition, the attached Xbasic code is used in my callback to create the report.

I am grateful for any help or guidance you can offer! Thank you!

[pcsystems]

Forgot to upload report component...

[mikeallenbrown]

Here is one option: Off load report generating to a second server and then deliver the PDF via email or Amazon S3. You'd need a runtime license from Alpha (and of course a small server).

I've developed several apps for various industries where there was some long running process that I needed to separate from the main app (so as not to bog it down). I use Amazon LightSail servers along with an Alpha Runtime license to automate/process all sorts of things: sending emails and SMS, creating large files, misc. database functions and maintenance, and large reports. It's my solution for things that the user doesn't need to see right away. My process then delivers the PDF, files, etc. via email or cloud storage.

The reports I generate are fairly straightforward text documents with no images so I can't really speak to your image troubles. One alternative to Alpha's report writer is another product called Stimulsoft. However it is quite expensive.

Lastly, when you submit a bug report make sure you submit the sample files, SQL tables if needed, etc. in a way that Alpha can easily duplicate the bug. I've had really good luck reporting bugs and find Alpha fairly responsive.

[madtowng]

AA has a report server option.
https://www.youtube.com/watch?v=XiWDrX0CagY

[pcsystems]

Thanks for your feedback on this!

Having additional servers seems like a great option long-term for large-scale use, however, during development as a proof-of-concept I need to come up with a single-server solution. Alpha's Report Server sounds like exactly what I need to accomplish, however, when I looked at that feature under Project Properties just now, it says it is not used when running under IIS or on Alpha Cloud...bummer...

As for submitting bug reports, yes, I agree Alpha has been very responsive...even to the point where Selwyn created a new function so we could quickly generate and download ZIP files containing a large amount of content. He's taught me quite a lot as I've worked on this project for the past few years. This thing with the image printing has been a tough one, apparently, but I'm confident there is a way to get there. As always, the trial-and-error and the learning process continues...

[madtowng]

I don't think the print server is an additional device.
I'm 99% certain it's a simple configuration setting but I'm waiting for confirmation from [email protected].

[madtowng]

I don't know if this is the answer we're looking for, but it seems to be easy enough to try.
Capture.png

[kkfin]

I wonder if Alphas new Document Templates feature is suitable also with pictures. You should test it.

[pcsystems]

Yes, thanks for the Doc Templates suggestion. I played around with that feature yesterday and found it doesn't present images in the correct orientation either. Today, I'm trying to see if I can get a viewbox to do what I need...

[pcsystems]

Just want to post this update for anyone who might have the same issues we had incorporating large numbers of images into PDF documents with Alpha. Our web app lets users upload any number of photos and videos from their mobile devices, with the option to create a single PDF showing all the content. In most instances, there are hundreds of files to process and trying to generate a PDF using Alpha’s report generator resulted in two main issues:

1. Some of the images would not be oriented properly in the PDF, depending on how the user had their mobile device oriented when the photo was taken. If the camera was in landscape mode, it would be oriented correctly. However, if a photo was taken in portrait mode or “upside down” in landscape or portrait, Alpha always rendered the image as if it was taken in "normal" landscape…meaning the image would appear sideways or upside down in the PDF.
2. If there were more than 30 or so image files, we started getting 413 (payload too large) errors from the AlphaCloud server.

Essentially, this meant we had to use other tools besides Alpha to accomplish what we needed…to print a PDF book or catalog containing a few hundred pages of photos if necessary.

We finally developed a solution that works well. Basically, all images in the final PDF must first be oriented properly and then compressed. To do that, we created an HTML template that would produce the desired layout and built the PDF one page at a time using Alpha’s HTML_TO_PDF function since it will correctly orient images automatically in an <img> tag. We then used a Python function to compress the images on that page and append the result to the final PDF as we went along.

For our testing, we used data that produced an uncompressed PDF of about 75Mb and ended up with a final compressed file of about 3.5Mb. Turns out, it’s all about the image compression algorithm. (Alpha’s HTML_TO_PDF function appears to have a compression setting, but we couldn’t get it to reduce our file size at all.) To see what is possible, we tried Adobe’s tool where you manually upload your PDF and have it compressed. Doing that converted our 75Mb file to around 256K with no noticeable loss of image quality! But we were looking for a standalone (and inexpensive) API that we could incorporate into our website.

We ended up using the free “Pypdf” Python library. In our testing, we tried different image compression settings and didn’t experience any noticeable image quality loss until around 10%. As a result, we settled on 25%, which gave us an acceptable 3.5Mb PDF file as output. Our largest live case so far resulted in a 145-page PDF totaling 171Mb, which is significantly lower than the 1Tb+ file it would have been originally. Note that it took about 15 minutes to generate the 145-page file, but with chunked responses we can show the user its progress. Also, if you know there will be a specific number of images per page in your app, you can reduce this time quite a bit. For us, there can be one or more images per page, depending on the content, so we need to account for that variable.

---------------------------------------------------------------------------------------------------------------------------------

Here’s the simple Python code to compress a PDF file:

#*** Compress PDF files by reducing image quality to 25 percent ***
#
# Need to install Pypdf package
#
def compress(e,session,request):

from pypdf import PdfReader, PdfWriter

in_file = e.dataSubmitted.FILEIN
out_file = e.dataSubmitted.FILEOUT

try:
reader = PdfReader(in_file)
writer = PdfWriter()

# count number of pages
totalPages = len(reader.pages)

for page in reader.pages:
writer.add_page(page)

for page in writer.pages:
for img in page.images:
img.replace(img.image, quality=25)

with open(out_file, 'wb') as f:
writer.write(f)
return "%s" % totalPages

except:
return "0"




Xbasic call:

...create the uncompressed pdf using HTML_TO_PDF...

e.dataSubmitted.FILEIN = CurrentPdfWorkFile ' uncompressed pdf
e.dataSubmitted.FILEOUT = WorkPdfFile1 ' compressed pdf
pythonresult = helper::Python::Call("pythoncomp","compress",e,session,request)
if pythonresult = "0" then
js = "$('reportmsg').innerHTML = 'Error writing page " + pcount + "';"
a5ajaxchunkedresponsewrite(js,e)
end if

...append the compressed pdf to the final file...

---------------------------------------------------------------------------------------------------------------------------------

If you have questions, I'm glad to help in any way I can...

[Pat Bremkamp]

Back in 2017, at a prior company, we created inspection reports which included a lot of photos taken by inspectors and customers out in the field and we would pick the ones we wanted and slot them into the possible 16 sections of the report. We standardized on 12 photos per page and resized the photos to fit into one of the 12 slots. I discovered that Alpha had a solution that let us use image magic for the photos, as in
a5_runimagemagiccommand(vFromPath+chr(92)+Name,vToPath+chr(92)+Name,"-resize x640")
This example moves a photo from one directory to another, resizes it to a max width of 640 pixels, and if Name was different, would rename it.

[pcsystems]

Thanks for your post, Pat! Yes, we use Alpha's imagemagic command to create thumbnails of each photo and I can see where it would work well for us if we had a fixed number of images per page. What led us to use a different approach was that each image can have from zero to 40 lines of descriptive information printed below it and we want to maximize how much can fit on a page in the resulting PDF. Also, users can upload any type of file (image, video, audio, PDF, html, text, etc) to our site, so the associated thumbnails can have different dimensions when they print. It's been fun cobbling all these tools together and realizing there are always multiple ways to get to the result we want. For me, I have to remember I never know how much I don't know, so it's a constant learning process and I appreciate your input!