Home Articles Categories Series
Pythonise Just now

Infinite scrolling & lazy loading with JavaScript & Flask - IntersectionObserver

Ditching pagination for smooth and infinite lazy loading, using the JavaScript IntersectionObserver API


Article Posted on by in JavaScript
Julian Nash · 9 months ago in JavaScript

Personally, I've never been a fan of pagination.

Asking users to paginate through content or to request more by clicking on a button works, but it's not very elegant.

Lazy loading content provides a more fluid user experience and increases the resposiveness of a page by only loading content when it's required. It's also a great way to keep users engaged as new content just "appears" once they reach a certain point in the page.

After some quick experimentation with the JavaScript IntersectionObserver, we were up and running with a lazy loading page within about 30 lines of code.

To be clear, this isn't "virtual scrolling" which can be achieved with the IntersectionObserver, however it's about half way there with the exception of not removing content from the dom.

The example

We're going to create a mock database of messages, featuring an index, title and some content.

On loading the page, the IntersectionObserver will trigger a function to request 10 messages which will then be rendered to the dom. Once the user reaches the bottom of those 10 messages, the function will trigger again and repeat the process until we're out of messages.

Server side

We're using Python & the Flask framework in this example but the same principles apply to any language or web server.

Python

from flask import Flask, render_template, request, jsonify, make_response
import random
import time

app = Flask(__name__)

heading = "Lorem ipsum dolor sit amet."

content = """
Lorem ipsum dolor sit amet consectetur, adipisicing elit. 
Repellat inventore assumenda laboriosam, 
obcaecati saepe pariatur atque est? Quam, molestias nisi.
"""

db = list()  # The mock database

posts = 500  # num posts to generate

quantity = 20  # num posts to return per request

for x in range(posts):

    """
    Creates messages/posts by shuffling the heading & content 
    to create random strings & appends to the db
    """

    heading_parts = heading.split(" ")
    random.shuffle(heading_parts)

    content_parts = content.split(" ")
    random.shuffle(content_parts)

    db.append([x, " ".join(heading_parts), " ".join(content_parts)])


@app.route("/")
def index():
    """ Route to render the HTML """
    return render_template("index.html")


@app.route("/load")
def load():
    """ Route to return the posts """

    time.sleep(0.2)  # Used to simulate delay

    if request.args:
        counter = int(request.args.get("c"))  # The 'counter' value sent in the QS

        if counter == 0:
            print(f"Returning posts 0 to {quantity}")
            # Slice 0 -> quantity from the db
            res = make_response(jsonify(db[0: quantity]), 200)

        elif counter == posts:
            print("No more posts")
            res = make_response(jsonify({}), 200)

        else:
            print(f"Returning posts {counter} to {counter + quantity}")
            # Slice counter -> quantity from the db
            res = make_response(jsonify(db[counter: counter + quantity]), 200)

    return res

The HTML

We're using the animate.css library for some simple animations when new messages come into view. We're also using the Bootstrap 4 library.

You'll also note the use of the HTML <template> tag.

HTML

<!doctype html>
<html lang="en">

<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">

  <!-- Bootstrap CSS -->
  <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css" integrity="sha384-ggOyR0iXCbMQv3Xipma34MD+dH/1fQ784/j6cY/iJTQUOhcWr7x9JvoRxT2MZw1T" crossorigin="anonymous">

  <!-- Animate CSS -->
  <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/animate.css/3.7.0/animate.min.css">

  <title>Infinite load</title>
</head>

<!-- sticky-top navbar displaying the counter -->
<nav class="navbar navbar-light bg-light sticky-top">
  <div class="container">
    <a class="navbar-brand" id="loaded" href="#">0 items loaded</a>
  </div>
</nav>

<body>

  <main class="mt-3">

    <div class="container">
      <div class="row">
        <div class="col">

          <h1 class="display-4 mb-3">Infinite load</h1>

          <!-- div to contain the content -->
          <div id="scroller" class="mb-3">

            <!-- template schema, hidden from the dom -->
            <template id="post_template">

              <!-- template content -->
              <div class="card mb-3 animated fadeIn shadow-sm">
                <div class="card-body">
                  <h4 class="card-title" id="title"></h4>
                  <span class="text-muted" id="content"></span>
                </div>
              </div>

            </template>

          </div>

          <!-- element to trigger the IntersectionObserver -->
          <div class="d-flex justify-content-center mb-3" id="sentinel">
            <div class="spinner-border" role="status"></div>
          </div>

        </div>
      </div>
    </div>

  </main>

  <!-- Import Bootstrap JS here -->

</body>

</html>

The key point to note in the HTML:

HTML

<div class="d-flex justify-content-center mb-3" id="sentinel">
  <div class="spinner-border" role="status"></div>
</div>

The sentinel div will be watched by the IntersectionObserver and will trigger a new request for data when it comes into the window view.

We've placed it at the bottom of the scroller div so when the user reaches the last of the messages, more content is loaded and appended to the dom.

The JavaScript

JavaScript

// Get references to the dom elements
var scroller = document.querySelector("#scroller");
var template = document.querySelector('#post_template');
var loaded = document.querySelector("#loaded");
var sentinel = document.querySelector('#sentinel');

// Set a counter to count the items loaded
var counter = 0;

// Function to request new items and render to the dom
function loadItems() {

  // Use fetch to request data and pass the counter value in the QS
  fetch(`/load?c=${counter}`).then((response) => {

    // Convert the response data to JSON
    response.json().then((data) => {

      // If empty JSON, exit the function
      if (!data.length) {

        // Replace the spinner with "No more posts"
        sentinel.innerHTML = "No more posts";
        return;
      }

      // Iterate over the items in the response
      for (var i = 0; i < data.length; i++) {

        // Clone the HTML template
        let template_clone = template.content.cloneNode(true);

        // Query & update the template content
        template_clone.querySelector("#title").innerHTML = `${data[i][0]}: ${data[i][1]}`;
        template_clone.querySelector("#content").innerHTML = data[i][2];

        // Append template to dom
        scroller.appendChild(template_clone);

        // Increment the counter
        counter += 1;

        // Update the counter in the navbar
        loaded.innerText = `${counter} items loaded`;

      }
    })
  })
}

// Create a new IntersectionObserver instance
var intersectionObserver = new IntersectionObserver(entries => {

  // Uncomment below to see the entry.intersectionRatio when
  // the sentinel comes into view

  // entries.forEach(entry => {
  //   console.log(entry.intersectionRatio);
  // })

  // If intersectionRatio is 0, the sentinel is out of view
  // and we don't need to do anything. Exit the function
  if (entries[0].intersectionRatio <= 0) {
    return;
  }

  // Call the loadItems function
  loadItems();

});

// Instruct the IntersectionObserver to watch the sentinel
intersectionObserver.observe(sentinel);

IntersectionObserver has many configuartion option which you can read about here over at the Mozilla developer docs.

In this example, we're calling the loadItems function if the intersectionRatio of the first entry is greater than 0. Which basically means "Do nothing if the sentinel element is not in view".

Uncomment the console.log(entry.intersectionRatio); section and scroll down to the bottom of the page to load more content. You'll see the values printed in the developer tools.

Wrapping up

After initial testing and playing around with the IntersectionObserver, we found it's easy to work with and setup an infitite, lazy loading feed of new content with very little code.

Next steps & testing will be to build a true virtual scrolling machanism, which dynamically inserts and removes elements from the dom depending on what's currently observable in the window.

Last modified · 24 Feb 2019
Did you find this article useful?
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License
Contents
Loading...