In our life, we usually take out smartphone to scan QR code. But when we are working on a computer, smartphone may not be the best choice for scanning QR codes that appear on web pages. One reason is the pictures taken from a monitor screen have the Moire' patterns which interfere QR code recognition. The other reason is you may want to use the decoded information directly on PC, such as a URL for opening a website. This article will implement a simple tool with Python to facilitate QR code recognition on desktop screen.
Installation
The required Python packages include PIL, OpenCV, Dynamsoft Barcode Reader and Qt.
python3 -m pip install pillow opencv-python dbr pyside2
Barcode SDK License
To unlock the capabilities of Dynamsoft Barcode SDK, you'd better apply for a 30-day free trial license.
Scanning QR Code from Screen
Since I have implemented a GUI barcode reader using Qt for Python, OpenCV and Dynamsoft Barcode Reader, the remaining stuff is to add screen snipping functionality. Inspired by https://github.com/harupy/snipping-tool, I learned the steps of implementing the screenshot functionality:
- Create a custom Qt widget and put it on top of the screen.
- Draw the selected area in
paintEvent()function while moving the mouse. - As the mouse is released, call
PIL.ImageGrab.grab()to get the image of the selected area.
Add buttons for snipping events
We open the design.ui file in Qt Creator and add two buttons for triggering the snipping events.
Save the file and recompile design.ui to design.py:
pyside2-uic design.ui -o design.py
In app_advanced.py, the two new buttons should be recognizable now. Connect them to slot functions:
self.ui.pushButton_area.clicked.connect(self.snipArea)
self.ui.pushButton_full.clicked.connect(self.snipFull)
Create a custom Qt widget
Create a SnippingTool.py file, in which we create a custom Qt widget:
import numpy as np
import cv2
from PIL import ImageGrab
from PySide2 import QtWidgets, QtCore, QtGui
from PySide2.QtCore import Qt
class SnippingWidget(QtWidgets.QWidget):
is_snipping = False
def __init__(self, parent=None, app=None):
super(SnippingWidget, self).__init__()
self.parent = parent
self.setWindowFlags(Qt.WindowStaysOnTopHint)
self.screen = app.primaryScreen()
self.setGeometry(0, 0, self.screen.size().width(), self.screen.size().height())
self.begin = QtCore.QPoint()
self.end = QtCore.QPoint()
self.onSnippingCompleted = None
def start(self):
SnippingWidget.is_snipping = True
self.setWindowOpacity(0.3)
QtWidgets.QApplication.setOverrideCursor(QtGui.QCursor(QtCore.Qt.CrossCursor))
self.show()
The size of the widget should be the same as the screen resolution, which can be obtained from the primaryScreen() function.
Next, we handle the mouse events:
def mousePressEvent(self, event):
self.begin = event.pos()
self.end = self.begin
self.update()
def mouseMoveEvent(self, event):
self.end = event.pos()
self.update()
def mouseReleaseEvent(self, event):
SnippingWidget.is_snipping = False
QtWidgets.QApplication.restoreOverrideCursor()
x1 = min(self.begin.x(), self.end.x())
y1 = min(self.begin.y(), self.end.y())
x2 = max(self.begin.x(), self.end.x())
y2 = max(self.begin.y(), self.end.y())
self.repaint()
QtWidgets.QApplication.processEvents()
self.close()
While the mouse is moving, we draw a rectangle to indicate the selected area in paintEvent() function:
def paintEvent(self, event):
if SnippingWidget.is_snipping:
brush_color = (128, 128, 255, 100)
lw = 3
opacity = 0.3
else:
self.begin = QtCore.QPoint()
self.end = QtCore.QPoint()
brush_color = (0, 0, 0, 0)
lw = 0
opacity = 0
self.setWindowOpacity(opacity)
qp = QtGui.QPainter(self)
qp.setPen(QtGui.QPen(QtGui.QColor('black'), lw))
qp.setBrush(QtGui.QColor(*brush_color))
rect = QtCore.QRectF(self.begin, self.end)
qp.drawRect(rect)
Take screen images
PIL is the Python library for capturing images from the screen. After releasing the mouse, we take the screenshot of the selected area based on the coordinates of the rectangle.
img = ImageGrab.grab(bbox=(x1, y1, x2, y2))
try:
img = cv2.cvtColor(np.array(img), cv2.COLOR_RGB2BGR)
except:
img = None
Taking full screen image is easy:
img = ImageGrab.grab(bbox=(0, 0, self.screen.size().width(), self.screen.size().height()))
Hide and show the application window
Once the snipping widget is ready, we can call it in button click event. Note: to avoid obstructing the screen, the application window should be minimized before launching the snipping widget and restored after the snipping is completed:
def onSnippingCompleted(self, frame):
self.setWindowState(Qt.WindowMaximized)
if frame is None:
return
frame, self._results = self._barcodeManager.decode_frame(frame)
self.showResults(frame, self._results)
def snipArea(self):
self.setWindowState(Qt.WindowMinimized)
self.snippingWidget.start()
def snipFull(self):
self.setWindowState(Qt.WindowMinimized)
self.snippingWidget.fullscreen()
Test the screen QR reader
-
Run the barcode recognition program:
python3 app_advanced.py Search Google for QR code.
-
Click the
Select Areabutton to scan QR code (one or multiple) returned by search engine.
You can also make a full screen barcode recognition by one click.
Top comments (0)