Behind our main web platform, we run a set of dedicated worker servers that process platform-generated jobs in the background. These jobs are usually generated by Connector actions, file import/exports and other automatic behaviors occurring on the platform. All such jobs are placed on a queue where they remain until a worker process picks up the job and executes it. These background jobs may sit on the processing queue for a little while before being processed.
Although usually jobs are picked up and processed within a few seconds, in some cases jobs can take longer to run, from several minutes to hours. There are several reasons why a job might take longer than expected:
- The job itself is a high resource-consuming request (e.g., a large data export). This simply takes time to run.
- There is a higher than normal job count in the queue due to peak loads which are generating many jobs at that point in time. We have auto-scaling approaches which respond to such loads, though there is a 10-30 minute lag while more servers spin up.
- There are a high number of jobs that require long processing times (e.g., very large PDFs). All jobs have a maximum run time of 120 seconds, so if there is a cluster of jobs taking, say, 100 seconds each, then this can backlog the queue.
- There are servers down for maintenance or being recycled, leaving fewer servers to handle the job flow. This is a normal activity that our team performs as needed.
Given the reasons above, we do not guarantee immediate execution of background jobs. Your jobs will get run by the system, just possibly not within a few seconds.
Form Connector Run Times
When a Form entry is uploaded to our platform, it is sent in two key parts:
- The answer data entered by the app user in the data fields
- Any media files (photos, videos, audio) captured during Form entry
The answer data from the Form is generally very quick to upload and does not require a fast network connection. Media files, however, are much larger files and thus take time to upload. Form Connectors that are based purely on the answer data will normally run almost immediately after the Form entry is uploaded; examples of these include Google Spreadsheet and HTTP POST Connectors. However, many other Connector types generate output files in PDF, Excel and Word formats. These Connectors require that all the media files (photos in particular) must be uploaded before they will run. This is to ensure that the output files generated do not have missing images. As such, you should not design a process that assumes Form Connectors will run as "live" or "immediate" activities.The more photos involved, the longer a Connector may delay since these can take a while to upload. This is especially true if your app users are working in areas with poor mobile data connections that have slow/intermittent upload speeds.
To complicate matters further, we must also consider the mobile operating system (OS). Every mobile OS tries to stop/kill the app as soon as possible once the app is no longer displayed. This is because a key priority of a mobile OS is to make sure that apps don't drain the device battery.
Reducing Delays In Form Connector Run Times
Since media file uploads are the key bottleneck, here are several ways to reduce delays:
- Make sure all your users are running the latest app version from the relevant app Store. We are always working to improve uploads and general performance of the app.
- Ask your users to try to keep the app open and in the foreground for a while after completing Forms. This will allow the app to have a longer, better chance to upload all the images. If they are not in an area with connection, then they will need to open the app again once they enter an area with connection for the app to resume uploading.
- Use fewer photos in your Forms. This can be as simple as advising your users to take fewer pictures, though you can also update Form designs to impose a maximum number of repeats in the repeatable photo capture pages.
- Change your Connector settings to force the sending of emails after a given timeframe. You can this so by setting the "Wait No Longer Than" option found under the Run Condition section of every Connector. This will force the Connector to run after your given maximum wait period, regardless of whether all pictures are uploaded.
If you continue to have problems with Connectors being heavily delayed in sending/running, please contact our support team.
Automatic Refresh Interruptions
When a Data Source connector that is set to automatically refresh ends up failing too many times for some reason, it will need to be manually restarted for the automatic refresh to resume. Connectors can be manually restarted by clicking "Run Now" on the individual connector within the Data Source's settings
page.
