Degrees of freedom

OpenLink Software Inc.'s newest iteration of the Virtuoso database engine is geared to take the wild innovation of its predecessor to new heights. Adding to Version 2.7's support for XML, Web services, and Internet standards -- a list longer than your arm -- the forthcoming Version 3.0 will also host ASP.Net pages and user-defined functions written in .Net languages on both Windows and Linux, more than proving it worth a look.

OpenLink CEO Kingsley Idehen, Virtuoso's mastermind, abides by two basic principles: No. 1, Data management is the foundation of IT; and No. 2, developers who build on that foundation need choices.

One dimension of choice is data flavor. To that end, Virtuoso's SQL core embraces XML, has a validating parser and XSLT (Extensible Stylesheet Language Transformation) processor embedded in it, can index and search within XML blobs, can express SQL queries as XML (optionally producing an XML Schema), and supports the still-evolving XQuery 1.0 standard.

Another dimension of choice is data access. In terms of data consumption, Virtuoso, which is rooted in OpenLink's suite of data-access drivers, can augment its own SQL tables and stored procedures by linking to foreign ones such as those from Oracle or Microsoft SQL Server. Its PL (procedure language) also includes primitives used to fetch data by way of HTTP Get, raw SOAP calls, or WSDL-described SOAP calls.

As a data provider, Virtuoso supports database-style clients with ODBC, JDBC, and OLE-DB drivers; SOAP clients by means of PL wrappers around local or remote procedures; and WebDAV (Web-based Distributed Authoring and Versioning) clients, including Windows Explorer and Office, by projecting file-system-like views of query results.

Yet another dimension of choice is platform -- a vague term that encompasses OS, application server, and programming environment. Virtuoso has many of the OS bases covered: Microsoft Windows, Linux, Sun Solaris, AIX, Hewlett-Packard HP-UX, and Mac OS X. It runs as a self-contained application server, using PL code embedded in Virtuoso Server Pages to manipulate the SQL, XML, and DAV resources controlled by the engine.

Whereas Version 2.7 is equipped with the option to embed a PHP (Hypertext Preprocessor) run time and a Java run time -- which can in turn host JSP (Java Server Pages) using, for example, Tomcat -- Version 3.0 extends hosting support to .Net, using ASP.Net on Windows, and to Mono -- the Ximian-led open-source .Net project -- on Linux.

But there's more to this story than just JSP or ASP.Net support. User-defined functions written in Java or .Net languages are directly available to SQL statements and PL procedures. Java or .Net objects can be serialized, stored in the database, and invoked from the database. Creating this object/relational/XML hybrid in a single environment is an accomplishment; doing it for Java and multiple flavors of .Net is truly inspiring.

Working Web services

There's so much going on under Virtuoso's hood that you really need a concrete example to see how the pieces fit together. Although it seems that everybody wants to write an RSS (Rich Site Summary) aggregator nowadays, I picked that project anyway because it nicely illustrates Virtuoso's diverse powers. The starting point for an RSS aggregator is a list of news feeds. My feeds live in a Frontier database and are published to the Web as an OPML (Outline Processor Markup Language) file. There are a dozen ways to use Virtuoso to acquire that list of feeds. I could, for example, use Virtuoso's built-in Web client to fetch the OPML file as a database column and then use its built-in XSLT processor to reduce the XML to a list of URLs. Instead, because I had already written a SOAP service that returns the list of feeds, I decided to call it from Virtuoso.

Radio UserLand doesn't produce WSDL wrappers for SOAP services, so I used Virtuoso's low-level SOAP client to retrieve the data as a PL vector -- essentially, a list of lists. Once I had a PL routine that completed this task, the data could in turn be wrapped up as a SOAP service. That enabled me to use Virtuoso's WSDL auto-generator to make the raw Radio service WSDL-consumable -- a neat bootstrapping trick! To pull this off, I wrote an XML Schema fragment that defines an ArrayOfString and fed it to Virtuoso's soap_dt_define function to define the new type. Finally, I used Virtuoso's auto-generated test harness to invoke the service through the WSDL wrapper.

The next step was to define the database schema for the news feeds. Because they are by definition valid XML files, I chose a simple two-column structure:

CREATE TABLE feeds (feedname VARCHAR, feeddata LONG VARCHAR)

For the BLOBs in the feeddata column, I created an XML-aware text index, like so:

CREATE TEXT INDEX XML on feeds (feeddata)

One way to populate this table would be to fetch the news feeds using Virtuoso's native Web client. Virtuoso PL is a capable scripting language, but it's nice to know that other, more familiar languages are an option. This was a great opportunity to try out the new C# support. On a Windows XP machine equipped with Version 1.0 of the .Net Framework, I wrote a C# method to fetch an URL and to ensure that its XML is well-formed. Then I compiled the method into the server's directory, used a PL procedure to import it, and used another PL procedure to invoke it once per feed.

Experiments in XML

On my first try I ran into a glitch in the beta software. Some XML files validated by the C# code were nevertheless rejected by Virtuoso's parser and could not be added to the database. OpenLink has since verified and fixed the problem, but it did serve to illustrate an interesting point. A C# (or Java) method called from a Virtuoso stored procedure is transacted, so the failures caused automatic rollbacks.

While OpenLink was analyzing the bug I found, I switched to the simpler Web-client approach to populate the database and try some XML queries. I've sometimes wondered about the distribution of RSS versions across the 100 news feeds I subscribe to; what's the breakdown among the RSS 0.9x, RSS 1.0, and RSS 2.0 versions? With the XML text index in place, I was able to select the union of two XPath-enabled queries -- one targeting the version attribute of RSS 0.9x and 2.0 feeds, the other isolating the RDF element of RSS 1.0 feeds -- and to group the results by frequency of version.

I could have piped these results through Virtuoso's XSLT processor and displayed them using its built-in Web server. But I decided to chart the data. To do that, I used Virtuoso's SQL-to-XML converter -- essentially the same as SQL Server's FOR XML AUTO data-shaping method -- to render the results as XML. I routed the results of the FOR XML query to Virtuoso's WebDAV repository, which can cache such results or regenerate them on demand. Then I pointed Excel 11 at Virtuoso's WebDAV server, imported the "file" into an XML data range, and charted it. Once it was all working, I repeated the same procedure on Red Hat Linux 8.0.

Think about the pathways involved here. Data moves from a SOAP service in Radio UserLand, through an auto-generated WSDL wrapper, into a database stored procedure, which calls out to the Web through a C# extension and stores results in an indexed XML database. Then an XPath-enabled SQL query gathers results, converts them to XML, and virtualizes them as a WebDAV resource, which Excel finally reads and analyzes. Too many degrees of freedom? Not to my way of thinking. "It's just data," says Apache officer and IBM senior developer Sam Ruby. Of course data takes many forms and flows in many ways. I can't think of another product that supports as many data pathways -- or does so as intelligently -- as Virtuoso.

Join the newsletter!

Error: Please check your email address.
Rocket to Success - Your 10 Tips for Smarter ERP System Selection
Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Jon Udell

Show Comments

Cool Tech

Breitling Superocean Heritage Chronographe 44

Learn more >

SanDisk MicroSDXC™ for Nintendo® Switch™

Learn more >

Toys for Boys

Family Friendly

Panasonic 4K UHD Blu-Ray Player and Full HD Recorder with Netflix - UBT1GL-K

Learn more >

Stocking Stuffer

Razer DeathAdder Expert Ergonomic Gaming Mouse

Learn more >

Christmas Gift Guide

Click for more ›

Most Popular Reviews

Latest Articles


PCW Evaluation Team

Edwina Hargreaves

WD My Cloud Home

I would recommend this device for families and small businesses who want one safe place to store all their important digital content and a way to easily share it with friends, family, business partners, or customers.

Walid Mikhael

Brother QL-820NWB Professional Label Printer

It’s easy to set up, it’s compact and quiet when printing and to top if off, the print quality is excellent. This is hands down the best printer I’ve used for printing labels.

Ben Ramsden

Sharp PN-40TC1 Huddle Board

Brainstorming, innovation, problem solving, and negotiation have all become much more productive and valuable if people can easily collaborate in real time with minimal friction.

Sarah Ieroianni

Brother QL-820NWB Professional Label Printer

The print quality also does not disappoint, it’s clear, bold, doesn’t smudge and the text is perfectly sized.

Ratchada Dunn

Sharp PN-40TC1 Huddle Board

The Huddle Board’s built in program; Sharp Touch Viewing software allows us to easily manipulate and edit our documents (jpegs and PDFs) all at the same time on the dashboard.

George Khoury

Sharp PN-40TC1 Huddle Board

The biggest perks for me would be that it comes with easy to use and comprehensive programs that make the collaboration process a whole lot more intuitive and organic

Featured Content

Product Launch Showcase

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?