Software Development

Converting UTZOO-Wiseman Usenet Tapes to Website with PostgreSQL backend using Python 3.8

Recently, I came across a resource that allowed me to download the entire collection of UTZOO NetNews Archive of the earliest USENET posts. These were essentially the earliest available discussions posted to the Internet by people working at various Universities who were already connected to the Internet. There were approximately 2.1 million posts in these archives created between Feb 1981 and June of 1991. This article describes the journey of converting those tapes into fully searchable PostgreSQL database and later also into the usenetarchives.com website.

[Read more…]

How to resolve TensorFlow 2.0 Error – Could not load dynamic library ‘cudart64_100.dll’

Recently I came across an issue while building ANN with TensorFlow 2.0, where IntelliJ complained about not being able to load the dynamic library ‘cudart64_100.dll’. The below instructions outline the approach for fixing the issue and would apply also to errors that refer to other NVIDIA GPU Computing Toolkit dynamic libraries, such as cufft64_100.dll, cufftw64_100.dll, cuinj64_100.dll, nppig64_100.dll,  nppim64_100.dll, nppist64_100.dll, nppisu64_100.dll, nppitc64_100.dll, npps64_100.dll, nppif64_100.dll, nppia164_100.dll or nppicc64_100.dll, all of which are part of the older version of CUDA platform for general computing on graphical processing units (GPUs) developed by NVIDIA . [Read more…]

Python: How to Create/Edit Windows Registries with Python 3.7

Just a short demo of how to go about creating/editing of Windows registry keys by using Python. This should be trivial, but as a matter of fact, not that many examples I could find online were working with Python 3.7. Below code should be pretty self-explanatory. Anyways, to put in in good use, let’s disable Cortana Assistant in Windows 10. If you have any questions about the process, just text me below.

[Read more…]

Visualizing JPEG Exif using Python: How to convert JPEG Meta-Data into Excel Travel Log and Google Map of All Visited Places

Over the past 20+ years, my family took tens of thousands of photos while visiting countries all around the world. As the years went by, the total number of stored images seems to grow every time we travel and even though we organize our images, sometimes it’s hard to recall when (exactly which year and month) we visited a specific place. Thinking about it, I realized that nowadays most photos are captured on smartphones or digital cameras that come equipped with GPS and automatically geotag each photo with GPS coordinates, as well as date and time of each image capture. That said, it shouldn’t be hard to convert the Exif information stored inside photo meta-data into easily readable Excel Travel Log, or too complicated to create a Google Map of all places I ever visited.

[Read more…]

How to use Python global keyboard hooks (& SQLite) to automatically add foreign accents to any text typed in Windows/Linux

The following article presents the programming solution to automatically add the diacritics to unaccented words typed using a regular English keyboard. There are solutions for Chrome browser or Firefox, but I wanted to have this option everywhere, in any program on any OS. Hence I decided to code it in Python. In this short article, I demonstrate how to use global keyboard hooks and SQLite database in Python to automatically add Slovak accents (diacritics) to text typed to any program running under Windows or Linux. This example can be easily used to adjust the program to any other foreign language. [Read more…]

How to query and visualize Apache logs in Splunk with Python

Splunk Enterprise is presumably one of the best platforms for gaining real-time intelligence from data. The following short tutorial explains how to install Splunk server (on Windows or Linux server) and how to use Python to programmatically query Splunk and visualize Apache data by plotting them on a graph by using the Matplotlib library. In this post, show step by step, how to add Apache logs to Splunk, and illustrate how relatively easy is to use Python to query how much bandwidth was consumed by one of my websites (in the past week, in megabytes). [Read more…]

Python Tutorial: How to create a Python GUI using Tkinter

The following is a short article on the basics of creating graphical user interfaces (GUI) in Python by leveraging Tkinter (interface to the Tk GUI toolkit). We’ll cover fundamentals such as creating a brand new window, centring the window on the screen, sizing it to desired dimensions, as well as giving the window a title and configuring it with its own icon. We also cover the basics of placing a new label and button on the window and wiring a new function that changes the text of our label with each button click. [Read more…]

Python Tutorial: How to Read-Write Excel Files, Web-Scrape Google and Create Interactive Maps in 20 lines of Code

The following short article shows just how simple it is to use Python programming language in a data science project. In this example, we’ll first go to Statista.com (public dataset provider) and download the MS Excel dataset that contains the list of 100 largest companies in the world. The file contains only two columns, the company name and their current market value. Our goal is to use Python to read the rows and cells inside the Excel file and use it to search the internet for some additional information, such as the company’s headquarters location and it’s map coordinates (latitude and longitude). You’ll see how easily this can be done by using Python web-scraping capabilities. We’ll also show how to write the newly found information back into the Excel sheet and use it to create an infographic that shows the headquarter location of 100 of the world’s top companies on the map.

[Read more…]

Converting UTZOO Usenet archive from magnetic tapes to MySQL database using Java

Recently, I came across a post from 2001 which allowed downloading the entire collection of UTZOO NetNews Archive of earliest USENET posts. These were essentially the earliest available discussions posted to the Internet by folks working for various Universities connected to the Internet. Millions of posts created between Feb 1981 and June of 1991.

Until 2001, these early Usenet discussions were considered being lost, but miraculously Henry Spencer from the University of Toronto, Department of Zoology was backing it up onto magnetic tapes and kept them stored for all these years (apparently at the great cost).

H. Spencer had altogether 141 of these magnetic tapes, but there were of no use, so eventually, him and a couple of motivated people such as David Wiseman (who dragged 141 tapes back and forth in his a pickup truck), Lance Bailey, Bruce Jones, Bob Webber, Brewster Kahle, and Sue Thielen; embarked on a process of converting all of these tapes into regular format, accessible to everyone.

And that’s the copy I downloaded. What a treasure, right?

Well, not so fast, once I unzipped the data, I realized that the TGZ format contains literally millions of small text files (each post in its own file). While it was certainly nice to have, it wasn’t something that I or anyone else could read. Certainly not in a forum like discussion format, it wasn’t obvious which post is the one that starts the discussion or which ones are the replies to the thread. And forget about searching through these files, that was utterly not possible. Just to put things into perspective, it took me over 5 hours to just unzip the archives.

That said, it didn’t take long for me to decide to develop a Java-based converter that would attempt to convert the entire collection from millions of flat files into a fully searchable MySQL database. The following post talks about the process and also includes the Java code of the solution released as open source.

[Read more…]

How create a custom browser URI scheme and C# protocol handler client that supports opening and editing of remotely hosted documents through WebDAV

Recently I came across a situation where I needed to create a one-click web-based solution, that allows clients to open and edit PhotoShop, AutoCAD, WordPerfect, PDF, LibreOffice and other files hosted remotely from my WebDAV enabled Apache web server by using their own desktop tools. Essentially I needed a solution for opening and editing remotely hosted documents (located on my web server) through a simple HREF link on a web page. This link, when clicked, would automatically open the PSD file hosted on my server, but do so in the PhotoShop installed on the client’s Windows computer, allowing them to seamlessly open and edit such files, without any need to download and upload the files back and forth to my server.  Initially, it seemed like an impossible, task, but then I realized that I should be able to create my very own custom browser URI scheme and some kind of a Windows-based client that handles opening of remote files in local installation Photoshop. After a bit of playing around with it, I got it working and created a solution capable of opening for remote editing not only PhotoShop files, but virtually any other remotely hosted file types in their associated applications, meaning in the software installed on the client’s desktop computer .  The following article is how I went about it. [Read more…]

Open Source Bananagrams Solver in JavaScript, PHP & MySQL

I had some free time and decided to find a programmatic solution to a game of Bananagrams… yeah, why not :)… So, I’ve built a variety of MySQL dictionaries from Aspell, HTML and CSS were used to style the page, then coded the backend in PHP and placed a solver logic into JavaScript web worker. Here is a short outline of the entire process along with a web site and YouTube demo. [Read more…]