How to scan store receipts with React

Automate receipt processing with React & receipt OCR (GitHub code included!)

Published in

Developer

Paper receipts mostly end up in the trash – they are hard to store for long periods of time, can’t be digitized easily, and formats differ across stores, locations, and regions.

What if we could turn a store receipt image, extract all the useful information out of it, and keep it in our records? Well, that's exactly what we’re going to do in this tutorial.

In this tutorial, we will:

Build a React frontend project with Chakra UI components
Let a user upload an image of a store receipt and pull out useful information such as item prices, store location, etc.
as JSON using Taggun API
Understand why browser-side environment variables do not keep API keys secret

Full source code: https://github.com/Hannahroseoc/taggun_react_turorial

Historical tutorial example. For current integration guidance, use the Taggun Quick Start. Keep API keys on your server; client-side environment variables do not keep them secret.

Also, you can check out how to build a backend here – How to Scan Receipts in NodeJS

Pre-requisites

Latest version of Node.js
Your favorite text editor, like Visual Studio Code
A basic knowledge of writing JavaScript code
An image of a receipt. Download a sample receipt (WebP)
API Key from Taggun

The Game Plan

Before we start building, here’s what are looking to achieve.

We will build a web app that lets a user select an image file to upload.

This can be an image of any store receipt. Once selected, it will be uploaded to the Taggun Receipt OCR API which converts the image to JSON data. All we have to do then is display it in a readable manner.

In a real-world scenario, you can use this same project to reward customers for the amount spent in-store, for receipt processing and data archiving purposes, or for simply converting non-structured data to structured data.

Getting started

Let’s start by creating a new React project.

Create React App takes care of generating a boilerplate project for us with a sensible file structure, a basic layout, and core packages – saving us a ton of time.

npx create-react-app taggun-frontend
cd taggun-frontend

‍

When the project is ready we can run it with:

npm start

‍

Open up http://localhost:3000 to see it running in your browser.

Awesome! We have a basic React app going.

‍

Add NPM packages

Let’s install some NPM packages to make our life easier.

npm i axios @chakra-ui/react @emotion/react@^11 @emotion/styled@^11 framer-motion@^6 @chakra-ui/icons @saas-ui/react

‍

Axios – Makes it easy to make a POST call to Taggun API
Chakra UI – Pre-built UI components
SaaS UI – Specialized UI components like Datatable and Loader for a loading animation

Once installed, let’s update the ./src/index.js file so we can use the brand new components from Chakra UI and SaaS UI:

import React from 'react';
import ReactDOM from 'react-dom';
import './index.css';
import App from './App';
import reportWebVitals from './reportWebVitals';
import { SaasProvider } from '@saas-ui/react'

ReactDOM.render(
  <React.StrictMode>
    <SaasProvider>
      <App />
    </SaasProvider>
  </React.StrictMode>,
  document.getElementById('root')
);

// If you want to start measuring performance in your app, pass a function
// to log results (for example: reportWebVitals(console.log))
// or send to an analytics endpoint. Learn more: https://bit.ly/CRA-vitals
reportWebVitals();

‍

Since it’s important the Provider is at the top-level file (index.js), all our React files will be able to correctly render the new components.

React UI – Upload image file

We are now ready to start building a UI that has:

1. Header section – explains what the project is and instructions on what the user should do next.

2. File upload section – a button to select a file from the computer and another to submit it.

I used a Ticket image as a logo in the Header section that you can download from here: https://github.com/Hannahroseoc/taggun_react_turorial/blob/main/public/outline.png

Make sure that you place it in the public folder for React to import it.

Update ./src/App.js like so:

import { Container, Center, Text, Box, Button, Image, VStack, Link } from '@chakra-
ui/react';
import { AttachmentIcon } from '@chakra-ui/icons'

function App() {

  return (
    <Container>
      <Box height="100vh" mt="30vh">
        <Box textAlign="center">
          <Center>
            <Image
              boxSize='45px'
              objectFit='cover'
              src='./outline.png'
            /></Center>
          <Text fontSize="xl" fontWeight="medium">Easiest way to scan your receipts using
Taggun API</Text>
          <Text fontSize="md" color="gray.500">Upload an image of a store receipt below to
see results</Text>

          <VStack mt={16}>
            <VStack spacing={6}>
              <Box>
                <label htmlFor="file-upload">
                  <AttachmentIcon /> Click to select receipt
                </label>
                <input id="file-upload" type="file" />
              </Box>
          </VStack>
            
            <Button size="md" colorScheme="primary">
              Upload photo
            </Button>

            <Link fontSize="xs" colorScheme="primary"
href="https://d7pdsiqo9rcig.cloudfront.net/wp-content/uploads/2019/04/receipt-
2.jpg">Download a sample one</Link></VStack>
        </Box>
      </Box>
    </Container>
  );
}

export default App;

Fantastic! Now we have a shell UI down that looks really good. The only problem is, it doesn’t do anything. Let’s fix that next.

Upload a file

We can use the onChange event on an input to listen to when a file has been selected by the user. The object is then is stored in a state variable.

<input id="file-upload" type="file" onChange={handleUploadFile} />

Update the ./src/App.js :

import { useState } from 'react';

const [file, setFile] = useState();

const handleUploadFile = (evt) => {
    setFile(
      evt.target.files[0],
    )
}

Add a Submit button

Similar to the input, a Button also has an event onClick that gets triggered when someone clicks it. We can use this to upload the file using Axios to Taggun.

Taggun expects a POST request with a file attached and an apikey as the header.

<Button size="md" colorScheme="primary" onClick={submitPhoto}>
              Upload photo
</Button>

Update the ./src/App.js :

import axios from 'axios';

const [file, setFile] = useState();
const [isUploading, setIsUploading] = useState(false);

...

const submitPhoto = async () => {
    if (!file) { return; }
    setIsUploading(true)
    const data = new FormData()

    data.append('file', file)

    let url = "https://api.taggun.io/api/receipt/v1/verbose/file";

    try {
      const res = await axios.post(url, data, {
        headers: {
          'Content-Type': 'multipart/form-data',
          'apikey': "API_KEY"
        } 
      })
      console.log(res)
      setIsUploading(false);
    } catch (e) {
      console.error(e);
    }
}

‍

Security note: this historical example sends requests directly from the browser. Do not put a real Taggun API key in client-side code. Use a server-side integration for production; see the Taggun Quick Start.

Run the project with npm start and try to upload the receipt image and press Submit. If everything goes well, you should see a JSON response in your console from Taggun.

Here’s what mine looks like:

React UI – Polish up

Now that we have data being returned from the API, we can polish up our UI so the user can easily interpret the data. Here’s how I chose to do it:

1. Show merchant’s address as Text

2. Show items sold in a Data Table

3. Show full response as JSON

I also added a loading animation for when we are waiting for the results to be processed.

import { Loader, DataTable } from '@saas-ui/react'
import { Container, Center, Text, Box, Button, Image, VStack, HStack, Divider, Link } from
'@chakra-ui/react';
import axios from 'axios';
import { useState } from 'react';
import { AttachmentIcon } from '@chakra-ui/icons'

function App() {
  const [res, setRes] = useState();
  const [file, setFile] = useState();
  const [isUploading, setIsUploading] = useState(false);
  const handleUploadFile = (evt) => {
    setFile(
      evt.target.files[0],
    ) 
  }

  const submitPhoto = async () => {
    if (!file) { return; }
    setIsUploading(true)
    const data = new FormData()
    data.append('file', file)

    let url = "https://api.taggun.io/api/receipt/v1/verbose/file";

    try {
      const res = await axios.post(url, data, {
        headers: {
          'Content-Type': 'multipart/form-data',
          'apikey': "API_KEY"
        }
      })
      console.log(res)
      setRes(res)
      setIsUploading(false);
    } catch (e) {
      console.error(e);
    }
  }
  return (
    <Container>
      <Box height="100vh" mt="30vh">
        <Box textAlign="center">
          <Center>
              <Image
              boxSize='45px'
              objectFit='cover'
              src='./outline.png'
            /></Center>
          <Text fontSize="xl" fontWeight="medium">Easiest way to scan your receipts using
Taggun API</Text>
          <Text fontSize="md" color="gray.500">Upload an image of a store receipt below to
see results</Text>
          <VStack mt={16}>
            {file ? <Box><Text>{file.name}</Text></Box> : <VStack spacing={6}><Box>
              <label htmlFor="file-upload">
                <AttachmentIcon /> Click to select receipt
              </label>
              <input id="file-upload" type="file" onChange {handleUploadFile} />
            </Box>
            </VStack>}
            {isUploading ? <Loader thickness="2px" /> : <Button size="md"
colorScheme="primary" onClick={submitPhoto}>
              Upload photo
            </Button>
}
            <Link fontSize="xs" colorScheme="primary"
href="https://d7pdsiqo9rcig.cloudfront.net/wp-content/uploads/2019/04/receipt-
2.jpg">Download a sample one</Link></VStack>
</Box>
        {res && <Box>
          <Box mt={10}>
            <Center>
              <Text fontWeight="medium">Successfully analyzed receipt</Text></Center>
            <Text fontWeight="medium">Details</Text>
            <Divider mb={4} />
            <HStack>
              <Text width={100}>
                Receipt #
              </Text>
              <Text>
                {res.data.entities.receiptNumber.data}
              </Text>
</HStack>

              <HStack>
              <Text width={100}>
Tax </Text>
              <Text>{res.data.entities.multiTaxLineItems[0].data.taxAmount.text}</Text>
            </HStack>
            <HStack>
              <Text width={100}>
                Merchant
              </Text>
              <Text>{res.data.merchantName.data}</Text>
              <Text>{res.data.merchantCity.text}</Text>
            </HStack>
          </Box>
          <Box overflowX="auto" mt={10}>
            <Text fontWeight="medium">Items</Text>
            <Divider mb={4} />
            <Box>
              <DataTable size="xs" columns={[{ "accessor": "data", "Header": "Currency" }, {
"accessor": "currencyCode", "Header": "Name" }, { "accessor": "text", "Header": "Item" }]}
data={res.data.amounts} />
</Box>

            <Box overflowX="auto" mt={10}>
              <Text fontWeight="medium">Full JSON response</Text>
              <Divider mb={4} />
              <pre style={{ "fontSize": "12px", "height": "400px", overflow: "scroll", }}>
                {JSON.stringify(res, null, 2)}
              </pre>
            </Box> </Box>
        </Box>}
      </Box>
    </Container>
  ); 
}
export default App;

Here’s what the UI looks like with the polish changes applied. Not too shabby right?

Environment Variables

This historical example uses a client-side environment variable. React embeds these values in the browser bundle, so a .env file does not keep an API key secret. For a production app, keep your Taggun API key on a server and send requests through your backend. The example below is retained for reference and needs updating before use.

‍

Create a new file called .env and add your own API Key:

REACT_APP_TAGGUN_API_KEY=123445 #replace with your own here

Now we can update the submitPhoto in ./src/App.js with:

headers: {
          'Content-Type': 'multipart/form-data',
          'apikey': process.env.REACT_APP_TAGGUN_API_KEY
        }

Congratulations! You’ve now built a fully functioning React project that can scan receipts and pull out important information such as items bought, price, tax amount, store name, etc.

Download the source code here – https://github.com/Hannahroseoc/taggun_react_turorial

‍