|
This is the first in a series of tutorials on using Talend Open Studio to perform data integration tasks. Talend Open Studio (TOS) is a very flexible tool for manipulating data - taking it from here and putting it there; but the documentation is a bit dry and it's sometimes difficult to figure out just which bits do what. This tutorial is taken from a real life task and demonstrates some of the more complex aspects of using TOS to perform a real job.
So what's the job? The requirements were these:
I've a weather station running at home (see my site TulligWeather). The software that logs the weather data writes the information to a "comma separated values" (CSV) data file - one line of data every 5 minutes. My requirement was to find a reliable way to insert this into a database so that I can manipulate it further (e.g. draw charts; but that's another tutorial). Once the data is in a database it's easy to query it to find maximum or minimum values, averages, trends and the like.
If you want to follow this tutorial then you'll need these ingredients:
- A MySQL database. I have MySQL installed on a separate (Linux) machine to my desktop; but you can install it on your Windows PC if you've got more than a Gb or so of RAM. Download the latest community version and run the installer. Make sure your remember your root password if you're prompted for one during installation. Many web hosting packages these days come with at least one MySQL database so you can use this instead of installing MySQL on your PC. If you want to use a web hosted database then you'll need to know the host name, user name and password for the database. Your web host should be able to help out with this information.
- Talend Open Studio. I've been using the latest release of version 4 which is at M3 at the time of writing. However I've found it to be perfectly stable and fixes an annoying bug that exists in version 3.2. Download the Windows installer, it's a fair size at more than 230Mb, and install it.
- Activestate Active Perl. This isn't strictly necessary - TOS can generate scripts in two languages, Java or Perl. Both of these are built into TOS; but the Perl is broken. You can do this whole project generating Java; but as I know Perl I prefer to use Perl. Once you've downloaded and installed Active Perl you'll need to run the Perl Package Manager and search for a package called DBD::MySQL. Mark this package for installation and then apply this to install the package. This will allow TOS to connect to your database using Perl.
Now you should have a MySQL database, TOS and Active Perl (including DBD::MySQL) installed on your PC. The next step will be to fire up TOS and create your project, more on this later.
Whilst I'm not a support service for MySQL, TOS or Active Perl installation let me know if you run into problems and I'll see if I can point you in the right direction.
|
|
|
For some days now it’s been speculated that IBM would purchase Sun Microsystems. However the news today that Sun is to be purchased by Oracle has far greater consequences than an IBM / Sun merger.
For a start Sun is a hardware manufacturer with a range of UNIX servers that target corporate IT infrastructure. Oracle, on the other hand, has no experience with box production and the corporate merger landscape is littered with companies that merged into areas they were not familiar with and came down to earth with a bump as a result. I would expect Oracle to hive off the hardware arm of Sun as a separate (and loss making?) business after which IBM can pick over the bones.
The rest of Sun Microsystems is a different prospect for Oracle. Sun own Java, the programming language and runtime that’s become pretty ubiquitous over the last ten years and can now be found on a huge range of every day devices from mobile phones, cars, washing machines; just about anything that needs a control system. Just how Oracle views Java could have far reaching consequences in a huge variety of market places.
Then there’s Sun’s other Open Source product set including the competitor for Microsoft Office; Open Office.org. Open Office has got to the point where it’s a serious alternative to the (very expensive) Office suite from Microsoft, and it won’t cost you a dime. Sun has been providing supporting services (if not hard cash) to the development of Open Office.org. Now I don’t know how much of a sink hole this has been for Sun; but it would be a great pity if Oracle abandon these efforts rather than continuing to promote and support development.
The MySQL database is another big question mark for Oracle. Purchased by Sun a few years ago MySQL could be seen as a direct competitor to Oracle’s database products, again at a fraction of the price. Removing MySQL from the market place could have been a big incentive for Oracle to purchase Sun. However the enterprise version of MySQL is only one part of the equation; the other part is what happens to the community version. For years now this has been the bedrock of many of the innovative open source projects that have made cloud computing what it is today. If Oracle should stop supporting the community edition of MySQL it will leave a huge hole in the open source development effort.
Down the line I predict this take over could have far reaching consequences for the shape of the IT industry. Let us know what your views are in the comments below.. |
 Imagine this scene. You come into work on a Monday morning to find that there was a fire that started in a neighbouring building over the weekend and your office was badly damaged by heat, smoke and water from the fire brigade. You're stood in the car park wondering what you're going to do to rescue your business.
|
|
|
I don’t know if it was all the bad press that Vista got; but something seems to have spurred Microsoft into the action in developing the next version of its Windows operating system. Currently called Windows 7 it’s been around in a beta version for a few months now and yours truly has been having a look.
|
|
|
|
|
|
