Instagram stories as a training format

Summary

Retail brands publish product stories on Instagram every day. Store staff watch them on their phones anyway. Today none of that content reaches the learning platform, and the product knowledge inside it evaporates after 24 hours. We propose a pipeline that imports every Reel and Story from a customer's connected Instagram account, detects the products in it, matches them to the product catalog and generates a training story: the video with tappable product tags, a product knowledge check, an outfit builder and an upsell round. The learner experiences it as a story. The admin experiences it as nothing: the content appears as a draft in Studio and in the programme.

Most of the building blocks exist. The Instagram Graph API gives us the media legally for connected business accounts. Our content API can create activities and place them in programmes. Our current templates cover the video, the quiz and the swipe round. Three things are new: a portrait story player with a product tag layer, an outfit builder template, and the import connector itself with its automation rules in Control.

The concept

A story is short, visual and already understood by every store employee. We keep that shape and add a learning layer behind it. Each imported story becomes one activity with three parts. First the video plays as a story: 9:16, progress bars, tap to skip, hold to pause. Products the pipeline detected show up as tags. Tapping one opens a product card with catalog facts and selling points. Second, training segments follow in the same visual language: a product check built from catalog facts, an outfit builder that answers a customer brief with pieces from the story and the wider range, and an upsell round where the learner swipes add-ons for a given basket. Third, the result syncs to the TinQwise activity like any other learning bite: completion, score, per-question analytics.

What the proof of concept shows

The proof of concept is a standalone web app, published alongside this document, built on four real Reels from the public @gstarraw account. It has two views. The learner view shows a phone with a story feed: three brand Reels (Utility in action, Denim and cargo, Greencast Denim) and one creator Reel (SHAPE Loose Jeans). Each plays as a story with product tags per scene, followed by a product check, and for the Greencast reel an outfit builder with a customer brief and budget and a five-card upsell round. The Control view shows the import queue with all six reels the scraper found, the connected sources, the automation rules and the pipeline log per item. A "Sync now" button retries the pending Yay Abe collaboration reel live. A "View generated activity JSON" button shows the payload the pipeline hands to Studio, in the shape of our real template schemas.

The fetch step ran for real through the scraper fallback: the Apify Instagram Reel Scraper returned six reels with caption, timestamp, play count, transcript and a downloaded copy of the video, for $0.32. Four videos were retrieved into the demo; two are shown as pending. Frame sampling and scene detection ran on the files. Product detection ran once on the sampled frames and captions and its output is embedded as data. Two things are illustrative. Prices and SKUs are placeholders, because no G-Star product feed is connected. The step timings in the pipeline log are estimates for the production setup, except the scraper run time, which is measured.

Two findings from the real material are worth keeping. Music-only reels produce useless transcripts ("The sky is clear.", "and water."), so the transcript add-on only pays off for talking-head content; the pipeline should decide per reel. And the creator reel is where the review queue earns its place: repost rights and opinion claims ("flattering") are exactly what a human should confirm before staff train on it.

Architecture

The pipeline is a chain of eight steps. Each step writes its output to the import record, so an admin can see exactly what happened and a failed step retries without redoing the rest.

flowchart LR
  IG[Instagram Graph API
Reels · video posts · Stories] -->|poll + webhook| F[1 Fetch media]
  AP[Apify scraper
fallback, owned accounts only] -.-> F
  F --> T[2 Transcode
H.264 720×1280, poster, audio]
  T --> S[3 Sample frames
1 fps, scene cuts, transcript]
  S --> D[4 Detect products
vision model, structured output]
  PIM[Product feed / PIM] --> M[5 Match catalog
SKU, price, attributes]
  D --> M
  M --> G[6 Generate training
tags · quiz · outfit brief · upsell]
  G --> Q[7 Quality gate
schema · confidence · claims]
  Q -->|≥ threshold| P[8 Publish draft
content API → Studio → programme]
  Q -->|below| R[Review queue in Control]
  R --> P

1. Getting the media

The primary source is the Instagram Graph API for Instagram Business and Creator accounts. The customer connects their account once through Meta's OAuth flow. From then on we read media on their behalf. Reels and video posts come from the media endpoint, with media_url, media_type, media_product_type (FEED, REELS, STORY), caption, permalink and timestamp. Stories come from the stories endpoint and are only readable for 24 hours after posting. Meta's story_insights webhook fires when a story expires, which is too late to fetch the file. We therefore poll connected accounts for new stories every 15 minutes. The rate limit of roughly 200 calls per user per hour leaves ample room. Media URLs are short-lived CDN links, so we download the file at fetch time and store it ourselves.

Production access needs Meta App Review for the read permissions (instagram_basic plus instagram_manage_insights for stories under Facebook Login, or instagram_business_basic under Instagram Login). Review takes one to four weeks and asks for a screencast of the use case. We plan that into the pilot and start with test users in development mode.

The fallback is a scraper such as Apify's Instagram scraper, for accounts the customer owns but has not connected yet. It works, and it is the fastest way to demo on a prospect's live account. It also sits outside Meta's terms, and the customer holds the rights to the content, not the account of a competitor. We restrict the fallback to accounts the customer declares as their own and log every use.

2 to 4. Understanding the video

We transcode every file to H.264 at 720×1280 for playback and take a poster frame. Landscape posts get a blurred fill behind them. Audio goes through a speech model so spoken product mentions become text. We sample one frame per second and run scene detection; in the demo clip that yields 16 frames and three scenes at 0, 6 and 11 seconds. Those scene cuts become the story segments and the natural moments for a question cue.

Product detection is one structured call to a vision-capable model with the sampled frames, the caption, the on-screen text from stickers and the transcript. The output is a JSON list of detected items: working name, category, visible attributes (colour, material, fit), the scenes and timestamps it appears in, a normalised position per scene for the tag, and a confidence. The model also returns styling context ("layered autumn look", "smart-casual") that the outfit builder uses, and any factual claims heard or shown ("waterproof", "100% wool") that the quality gate checks against the catalog.

Cost is small. Sixteen frames at reduced resolution plus text is well under 30k input tokens, so one Reel costs cents, not euros. The bigger cost driver is review time when confidence is low, which is why catalog matching matters.

5. Catalog matching

Detection tells us "camel wool coat". The catalog tells us it is SKU NR-CT-2201, €189, 70% wool, dry clean only, in stock in Utrecht. Only the second version is worth training on. We match detected items against the customer's product feed. Most retailers already maintain a Google Shopping or Meta Commerce feed with title, description, image, price, category, colour, material and availability. That feed is our first integration; a PIM connector comes later. Matching uses text embeddings on title and attributes plus image similarity on the product image, and returns the top candidates with a score. One clear winner above threshold matches automatically. Two close candidates (the sneaker in the demo, white versus off-white) send the item to review with the candidates side by side, so the admin picks in one click.

Without a feed the pipeline still works. It generates training from what is visible and said, with a lower confidence and a "provisional" marker on the product card. Customers see the difference within a week and connect the feed.

6. Generating the training

Generation is a second model call, grounded strictly in catalog facts and the story context. The prompt forbids claims that are not in the feed. Output is one activity in our template shapes, in the platform's authoring language, with translations to the other platform languages as a second pass. Four kinds of training come out of one story. Product tags: position, time window and SKU per detected item. Product check: three to five questions per story, mixing text questions on facts (material, care, fit, price) and one image question (which item costs X). Outfit brief: a customer persona, a budget, required categories and a pool that includes the story's items plus related items from the feed, with rules for the verdict. Upsell round: the hero product as basket, four to six candidate add-ons with a yes/no and a one-line rationale grounded in the feed and the story.

Every question carries feedback for both outcomes, because that is where the learning happens. The demo's feedback texts show the tone we aim for: short, specific, usable on the shop floor.

7 and 8. Quality gate and publishing

The gate has three checks. Schema validation against our template schemas, so nothing invalid reaches Studio. Confidence: the weighted average of detection and matching scores must exceed the customer's threshold, 0.85 by default. Claims: any claim in the generated copy must trace back to a feed attribute; anything else is removed or flagged. Items that pass are created through the content API (create_activity, edit_activity_draft, add_activity_to_section) and placed in the programme and section the admin chose in the rules. Items that fail wait in the review queue with the reason.

The story player

Our interactive-video template plays landscape video with question cues. Stories are portrait, tap-driven and layered. We build the story player as a presentation mode over the existing activity model, not as a new content type. The activity stays a learning bite with slides; the player renders the video slide as a story with progress bars per scene, tap zones, hold to pause, and a tag layer reading positions and time windows from the activity. Training slides render as story segments with the same header and progress bars. The template gets three additions: portrait support and a higher file size limit, a product_tags collection, and a presentation.player = "story" flag. Existing landscape activities keep working unchanged.

Fit with current templates

The table maps each element of the format onto what Studio offers today. Green exists and is used as is. Amber exists and needs an extension. Red is new.

The generated activity in the proof of concept follows these shapes. The JSON view in the Control tab shows an interactive-video slide with cues, an mpc-image-validated slide and a swipe-series slide exactly as our schemas define them, plus the proposed outfit-builder marked as new.

Data model

Six entities carry the pipeline. They live in the platform database next to activities, scoped per platform like everything else.

Learner results need no new entities. Completion, scores and per-question analytics land on the activity as they do today. One new dimension is worth adding to analytics: the SKU. When we know which product a question was about, we can report product knowledge per store and per product, and we can generate a refresher automatically when a product scores low.

Admin experience

The admin does three things once. Connect the Instagram account through Meta's login. Paste the product feed URL. Choose a programme and section for new stories. Then five rules run the format, all shown in the proof of concept: import new Reels and Stories; skip content without products; auto-publish above a confidence threshold; attach to a programme; generate in all platform languages. The import queue in Control shows every piece of media the pipeline looked at, including the ones it skipped and why. That transparency is what makes admins trust automation. Items in review show the reason and the one decision needed, such as picking between two SKUs.

Expected admin time after setup: a few minutes a week for review items, zero for everything else. The proof of concept shows this as a counter on purpose.

Risks and compliance

Effort and phasing

Estimates assume two developers plus part-time design and a product owner. Weeks are elapsed development weeks.

A third phase closes the loop: refreshers generated automatically for products that score low per store, and the same pipeline pointed at other video sources the customer owns, such as TikTok, YouTube Shorts and internal campaign videos. The architecture above is source-agnostic from step 2 onward, so that phase is mostly connectors.

Decisions we need

Pilot customer

One fashion retailer with an active Instagram Business account and a product feed. The demo uses a fictional brand so we can show it to anyone before we choose.

Scraper fallback: yes or no

It speeds up demos and onboarding, and it carries terms-of-service risk. Proposal: keep it internal for demos, never ship it as a customer-facing option.

Auto-publish permission

Building the server-side save changes a safety assumption in Studio. We propose it as an explicit per-platform permission the customer switches on.

Outfit builder scope

Fashion first, or generic mix-and-match from day one. The generic version costs two extra weeks and opens food, drinks, tech and interior customers.

Sources