Author: randomds

Use Python Selenium with Snap browsers

TL, DR Selenium is one of the main libraries for browser automation and web scraping. Sometimes it is painful to integrate with browsers that are installed as Snap packages, this guide gives you a few examples for a correct configuration. What are Snap packages? Snap packages is the new and preferred way to distribute applications… Continue reading Use Python Selenium with Snap browsers

Set your User Agent with Python Requests, Scrapy, and Selenium

TL, DR When you crawl the web to collect data you should set a User Agent that identifies you. Or one that hides the tool you are using. Here you can find how to set the User Agent in Python Requests, Scrapy, and Selenium. What is the User Agent? A User Agent is a string… Continue reading Set your User Agent with Python Requests, Scrapy, and Selenium

Deploy Minio with Docker on a Synology NAS with a custom domain name

TL, DR Minio is a powerful open source self-hosted object storage, similar to S3, that you can deploy on your own hardware or cloud instance. This guide provide you the right settings to deploy Minio with Dover on a Synology NAS and use your domain name with SSL Minio object storage We already covered Minio… Continue reading Deploy Minio with Docker on a Synology NAS with a custom domain name

MongoDB aggregation: save results in a Pandas DataFrame

TL, DR MongoDB is one of the leading NoSQL databases, and its aggregation framework enables powerful queries, as well as data operations. We will see how to save results from aggregation pipelines into a Pandas DataFrame. From MongoDB to Pandas I already provided an introduction to MongoDB and Compass in a previous post for my… Continue reading MongoDB aggregation: save results in a Pandas DataFrame

MongoDB aggregation: match a field with values in a list

TL, DR MongoDB is one of the leading NoSQL databases, and its aggregation framework enables powerful queries, as well as data operations. We will see how to match a field with values in a list to help you select. Matching values in a list I already provided an introduction to MongoDB and Compass in a… Continue reading MongoDB aggregation: match a field with values in a list

MongoDB aggregation: create your data pipelines

TL, DR MongoDB is one of the leading NoSQL databases, and its aggregation framework enables powerful queries, as well as data operations. We will see how to quickly create data pipelines and create code you can use in your projects. The power of a Compass You use MongoDB as data repository because it’s so simple… Continue reading MongoDB aggregation: create your data pipelines

Copy files with specific naming patterns from Linux shell

TL, DR Sometimes you may need to copy a large number of files, and your file explorer/file search will be painfully slow. If you use Linux, you can easily copy files with a specific naming pattern using only the shell, saving time and effort. Need to copy many files? Sometimes we need to copy a… Continue reading Copy files with specific naming patterns from Linux shell

Fetch and display an image with PyScript

TL, DR PyScript is a new and cool piece of technology that let you use Python in the browser. In this tutorial we will create a webpage that fetch and display a cat image. More tutorials on PyScript to come! Use PyScript to fetch and display an image Python is famous for running everywhere. Jupyter… Continue reading Fetch and display an image with PyScript

Alpha Vantage API: Free financial markets data

TL, DR Alpha Vantage is a free API where you can get financial markets data. It provides you stocks, Forex, and cryptocurrency data feed, as well as many technical and economic indicators. You can get a free API key at this link. You can get higher limits for API utilization by holding Alpha Vantage Coins… Continue reading Alpha Vantage API: Free financial markets data