Python Tutorial: How to Read-Write Excel Files, Web-Scrape Google and Create Interactive Maps in 20 lines of Code

The following short article shows just how simple it is to use Python programming language in a data science project. In this example, we’ll first go to Statista.com (public dataset provider) and download the MS Excel dataset that contains the list of 100 largest companies in the world. The file that contains only two columns, the company name and their current market value. Our goal is to use Python to read the rows and cells inside the Excel file and use it to search the internet for some additional information, such as the company’s headquarters location and it’s map coordinates (latitude and longitude). You’ll see how easily this can be done by using Python web-scraping capabilities. We’ll also show how to write the newly found information back into the Excel sheet and use it to create an infographic that shows the headquarter location of 100 of the world’s top companies on the map.

[Read more…]

Converting UTZOO Usenet archive from magnetic tapes to MySQL database using Java

Recently, I came across a post from 2001 which allowed downloading the entire collection of UTZOO NetNews Archive of earliest USENET posts. These were essentially the earliest available discussions posted to the Internet by folks working for various Universities connected to the Internet. Millions of posts created between Feb 1981 and June of 1991.

Until 2001, these early Usenet discussions were considered being lost, but miraculously Henry Spencer from the University of Toronto, Department of Zoology was backing it up onto magnetic tapes and kept them stored for all these years (apparently at the great cost).

H. Spencer had altogether 141 of these magnetic tapes, but there were of no use, so eventually, him and a couple of motivated people such as David Wiseman (who dragged 141 tapes back and forth in his a pickup truck), Lance Bailey, Bruce Jones, Bob Webber, Brewster Kahle, and Sue Thielen; embarked on a process of converting all of these tapes into regular format, accessible to everyone.

And that’s the copy I downloaded. What a treasure, right?

Well, not so fast, once I unzipped the data, I realized that the TGZ format contains literally millions of small text files (each post in its own file). While it was certainly nice to have, it wasn’t something that I or anyone else could read. Certainly not in a forum like discussion format, it wasn’t obvious which post is the one that starts the discussion or which ones are the replies to the thread. And forget about searching through these files, that was utterly not possible. Just to put things into perspective, it took me over 5 hours to just unzip the archives.

That said, it didn’t take long for me to decide to develop a Java-based converter that would attempt to convert the entire collection from millions of flat files into a fully searchable MySQL database. The following post talks about the process and also includes the Java code of the solution released as open source.

[Read more…]

How to take screenshots using Flameshot in KDE Plasma Desktop (Ubuntu 18.10) and associate it with the PRINT (PRT SC) keyboard shortcut

As SnagIt currently doesn’t support Linux, the Flameshot is likely the only screenshot taking utility worthwhile to consider if you’re a Linux user. Here are step by step instructions on how to install Flameshot under Kubuntu KDE Plasma Desktop and also how to associate it with the PRT SC button.

[Read more…]

How create a custom browser URI scheme and C# protocol handler client that supports opening and editing of remotely hosted documents through WebDAV

Recently I came across a situation where I needed to create a one-click web-based solution, that allows clients to open and edit PhotoShop, AutoCAD, WordPerfect, PDF, LibreOffice and other files hosted remotely from my WebDAV enabled Apache web server by using their own desktop tools. Essentially I needed a solution for opening and editing remotely hosted documents (located on my web server) through a simple HREF link on a web page. This link, when clicked, would automatically open the PSD file hosted on my server, but do so in the PhotoShop installed on the client’s Windows computer, allowing them to seamlessly open and edit such files, without any need to download and upload the files back and forth to my server.  Initially, it seemed like an impossible, task, but then I realized that I should be able to create my very own custom browser URI scheme and some kind of a Windows-based client that handles opening of remote files in local installation Photoshop. After a bit of playing around with it, I got it working and created a solution capable of opening for remote editing not only PhotoShop files, but virtually any other remotely hosted file types in their associated applications, meaning in the software installed on the client’s desktop computer .  The following article is how I went about it. [Read more…]

How to Disable Viber Annoying Ads on Windows and Macintosh

I don’t mind advertising, as long as it remains unobtrusive to the end user. As a matter of fact, I am using ads on this blog. However, just recently Viber implemented the new type of video ads implemented on the Windows and Macintosh desktop Viber platform. These ads aren’t your usual standalone picture ads, these are video ads that tend to auto-play on their own along with the sound, which is especially annoying when one forgets to turn off their PC speakers. Unfortunately, Viber does not provide application settings to disable or control ads. So, here is how you can easily disable these ads on your Windows or Mac machine.

[Read more…]

Open Source Bananagrams Solver in JavaScript, PHP & MySQL

I had some free time and decided to find a programmatic solution to a game of Bananagrams… yeah, why not :)… So, I’ve built a variety of MySQL dictionaries from Aspell, HTML and CSS were used to style the page, then coded the backend in PHP and placed a solver logic into JavaScript web worker. Here is a short outline of the entire process along with a web site and YouTube demo. [Read more…]