JSON, Level: Advanced, Version: FM 19.6 or later, Virtual List

Virtual List Even More Simplified

Demo file: virtual-list-even-more-simplified.zip  [expects, but does not require, FM 22 or later]

Note 1: The example in today’s article/demo is intentionally very basic.
Note 2: The demo is self-populating to keep the data current, so the values you see in the screen shots may not exactly match those you encounter in the demo.

As long time readers may recall, between 2011 and 2019 this blog featured numerous articles touching on various aspects of the virtual list technique — or, more properly, series of techniques. The following year, in 2020, I decided to distill the core bits into a basic “beginner’s guide” with accompanying demo file, and I called it Virtual List Simplified. And if you’re still using FM 21 or earlier, and are looking for an introduction to virtual list, I highly recommend that article.

However, with the release a few months ago of FM 22 (a.k.a. FileMaker 2025), the landscape dramatically changed thanks to a re-written JSON engine that is significantly more powerful (i.e., faster) than before. Why should that matter? Because JSON is typically where virtual list gets its data from, and thanks to this awesome performance boost, it’s time to recycle and update the 2020 article and accompanying demo file to take advantage of it. And good news: it’s now less complicated and easier to understand.

At the risk of stating the obvious, there are many, many ways to skin the virtual list cat, and the purpose of today’s article is not to say “this is the best way”, or imply that other approaches are flawed, but simply to propose one particular approach you might take — especially if you are either: a) new to virtual list, or b) already using virtual list, but aren’t completely happy with your current implementation.

At any rate, my aim today is to gather useful insights from earlier articles into a single document (with a couple new ideas thrown in), and some of what follows has been recycled from those earlier articles.

Spoiler alert: two-dimensional JSON arrays are now viable and performant as back-ends for virtual list. Prior to FM 22 this was not the case… things would appear to be fine when the 2D array was relatively small, but would slow down dramatically with larger 2D arrays. For details see Virtual List with a 2D JSON Array in “FileMaker Pro 22: Initial Impressions”.

Continue reading “Virtual List Even More Simplified”

JSON, Level: Advanced, Version: FM 16 or later

Fast Summaries to JSON, part 2

This is a follow up to last week’s Fast Summaries to JSON and will assume the reader is familiar with that material.

Demo Files

After last week’s article appeared, a reader contacted me wondering whether the technique could be adapted to provide a more literal JSON representation of what was visible on-screen. The answer turns out to be yes, with a bit of additional work…

Continue reading “Fast Summaries to JSON, part 2”

JSON, Level: Advanced, Version: FM 16 or later

Fast Summaries to JSON

The other day some colleagues were discussing a need to produce a JSON representation of the data in a summary report. Long time readers of this blog will know that I am obsessed with fond of the Fast Summary technique, and it seemed like it might be a good fit for this challenge, so I built some demos to find out if that would be the case.

Demos 1 and 4 will work with FM 16 or later. Demos 2 & 3 utilize recently-added JSON features, so require FM 21 or later.

Continue reading “Fast Summaries to JSON”

JSON, Level: Advanced, Summary List, Version: FM 18 or later

Render Found Set as JSON Object

Demo file: found-set-to-json-object.zip (40MB; requires FM 18 or later)

[Author’s note: the demo file is a work in progress. Modify it as you see fit.]

Today we’re going to compare various methods one might consider employing to render an arbitrary found set as a JSON object. The demo consists of a table of US customers with 250K records, and the JSON object will be structured like so…

{ 
   State Name: 
      { County : [ [ customer array ] , [ customer array ] ... ] , 
        County : [ [ customer array ] , [ customer array ] ... ] , 
        ...
      } , 
   State Name:  
      { County : [ [ customer array ] , [ customer array ] ... ] , 
        County : [ [ customer array ] , [ customer array ] ... ] , 
        ...
      } , 
   ...
}

…i.e., grouped by state name and county, with customer data represented as a two-dimensional array, for example: Continue reading “Render Found Set as JSON Object”

JSON, Level: Advanced, Version: FM 16 or later, Virtual List

Virtual List Simplified

Demo file: virtual-list-simplified.zip

Note 1: The example in today’s article/demo is intentionally very basic.
Note 2: The demo is self-populating to keep the data current, so the values you see in the screen shots will not exactly match those you encounter in the demo.

Recently I had the pleasure of discussing virtual list with Paul Jansen and Jeremy Brown on The Context podcast. One consequence of having written so much on the subject over a period of many years, is that information has been spread across many articles. Another consequence is that my thinking re: certain implementation specifics has changed over time.

At the risk of stating the obvious, there are many, many ways to skin the virtual list cat, and the purpose of today’s article is not to say “this is the best way”, or imply that other approaches are flawed, but simply to propose one particular approach you might take — especially if you are either: a) new to virtual list, or b) already using virtual list, but aren’t completely happy with your current implementation.

At any rate, my aim today is to gather some useful insights from earlier articles into a single document (with an occasional new idea thrown in as well), and some of what follows has been recycled from those earlier articles. Continue reading “Virtual List Simplified”

Level: Intermediate, Version: FM 13 or later

Fast Summaries Re-revisited

My recent “Virtual List on Steroids” presentation at DIG-FM, and the preparation of a revised version to give at dotFMP next week, has prompted me to reconsider certain assumptions re: optimizing performance both locally, and across a LAN and/or WAN.

2018-05-29_094509

Specifically, there are two things we’re going to look at today. First, a way to dramatically speed up sorting on related data. Then, having incorporated that trick into the Fast Summary approach, we will compare Fast Summaries vs. Multi-Finds under various scenarios.

Demo files (structurally identical):

Continue reading “Fast Summaries Re-revisited”

ExecuteSQL, Level: Intermediate, Version: FM 13 or later

Fast Summaries Revisited

Update 30 May 2018: see also Fast Summaries Re-revisited.

If you do complex data analysis, then from time to time you probably need to group, summarize, and parse data into variables and/or fields. There are various ways to accomplish this, including the Fast Summary technique, which regular readers of this blog may be familiar with, as it has made a number of appearances here over the years.

Well today we’re going to look at a couple alternatives to Fast Summaries, with the help of some demo files, which you can use to do your own benchmarking.

2016-03-29_223920

Or, if you’d rather not do your own benchmarking, scroll down to see results from my tests. Bottom line: each of these three techniques can be fastest, depending on circumstances.

(Also, if you’ve been putting off exploring Perform Script On Server, a.k.a. PSOS, host the demo files on FileMaker Server 13 or later. The reporting routine optionally uses PSOS, so you can dive in painlessly and see what you’ve been missing.) Continue reading “Fast Summaries Revisited”

Chart, Level: Advanced, Version: FM 13 or later, Virtual List

FM 13: Virtual List Charts, part 2

Picking up where we left off in part 1, today we’re going to take a look at examples 2 through 6 in the Virtual List Charts demo file (the demo has been updated since part 1, so I recommend downloading a fresh copy).

5-16-2014 4-08-08 PM

We covered example 1 and most of the general concepts last time, so today we’re mainly going to touch on specific points of interest, but to briefly recap… Continue reading “FM 13: Virtual List Charts, part 2”

Chart, Level: Advanced, Version: FM 13 or later, Virtual List

FM 13: Virtual List Charts, part 1

4-20-2014 11-18-24 AMToday we’re going to look at applying the virtual list technique to FileMaker charting with the goal of producing a reusable chart “object”, or rather, a series of chart objects. We’ll need more than one because while certain attributes (e.g., chart title) can be set programmatically, others, including type (e.g., column or line), must be hard-coded into the chart object.

We’ve already explored Bruce Robertson’s virtual list on this site a number of times, but briefly, you create a utility table in your solution to facilitate non-standard viewing, reporting, etc., and pre-populate it with “more records than you’ll ever need”. The records in this table will derive their data “virtually”, by parsing it from an array — typically one or more $$variables.

Well it turns out the technique can be applied to charting as well, and today we have a demo file, Virtual List Charts, that contains six examples: three for Web Visits…

4-20-2014 2-03-32 PM

Continue reading “FM 13: Virtual List Charts, part 1”

Level: Intermediate, Version: FM 10 or later, Virtual List

Conditional Subsummary Report in Browse Mode

Last time we looked at several summary reporting tricks, including a conditional subsummary (when an item’s Status is “Scheduled” it will have a value in the Substatus field — otherwise Substatus will be blank). The challenge was to generate a summary report showing Substatus only where appropriate, without seeing any annoying empty gray rows beneath Pending, Cancelled or Completed. And last week’s report worked fine… in preview mode.

9-11-2013 10-43-07 PM

Continue reading “Conditional Subsummary Report in Browse Mode”