Feeds SQL – getting arbitrary data Drupalized….

Feeds SQL does its job so well it’s rather mindblowing.   As the name suggests it allows Feeds to use a SQL query as its data source.  The mapping and all the rest of it works as one might expect, and that’s what is blowing my mind right now.  After installing the module

SQL Fetcher

SQL Fetcher

Firstly set your fetcher to the sql fetcher…  I only have one database in my settings file for now, but that’s sure to change.

Secondly choose the SQL Parser… I guess this means that you might be able to upload a file in to the fetcher and then parse it… for this example we’re going SQL Fetch and SQL Parse though.

Choose the SQL Parser

Choose the SQL Parser

And now the magic begins…  in the settings you can test your sql fetch by putting in a select statement.  For this example I made a dummy table with some random stuff in it

SQL Parsing

SQL Parsing

and now comes the wow!

It's data!

It's data!

And more wow!

Your columns are ready to be mapped!

Your columns are ready to be mapped!

So pretty – and just like that all those whacked databases you were trying to decide how to merge in to Drupal are available with feeds…. yum yum yummy. Mad props to vkareh for the dev on this piece… absolutely awesome.

This entry was posted in Planet_Drupal and tagged , , , . Bookmark the permalink.

10 Responses to Feeds SQL – getting arbitrary data Drupalized….

  1. Looks like a nice feature, but I think I’m missing the ‘wow.’ Run query, display results… what’s so amazing?

    • alibama says:

      that is exactly it – frankly i’ve had to patch feeds so many times for varying processes that when it works as expected I’m pretty amazed. take the generic entityform processor for instance – that thing should have been committed last year and instead I’m patching it still… ditto working with organic groups – always having to dig around for something that seems like it should work… this is one of those fine times when feeds does exactly what is expected of it…

  2. Fabianx says:

    There is also:

    http://drupal.org/project/feeds_db

    That has the advantage that it allows chunking of the data via batch API, so you don’t need enormous amounts of memory.

    We had created that module without knowing of feeds_sql, but it still has some unique features that could / should be merged into feeds_sql.

    Thanks,

    Fabian

    • alibama says:

      wow – thanks man, hadn’t seen that one – will be glad to test it, am particularly keen on getting in to an oracle database that we have with financial data – hopefully one of them works

      • Fabianx says:

        We had trouble getting that to work with Oracle DB.

        In the end we used a little python script to create CSV files, which we re-imported as MySQL.

        Then changed some columns from text back to their original schema and added some indexes and it worked out quite well.

        If you need the script drop me a mail.

        Best Luck with that,

        Fabian

  3. Andrew Morton says:

    I just started using this module as well. Here’s some tips:

    If you need to enter some value into the new nodes, you could of course use HOOK_feeds_presave() in a module… But since I just needed “US” to put into the country address field, my query is:
    SELECT *, 'US' as country

    I also needed to generate a unique ID, which I did by combining 2 fields:
    CONCAT(`fscs_key`, '-', `fscs_seq`) AS guid

  4. Urlop says:

    Hi, thanks for post. Do you know does it work in Drupal 6?

    • alibama says:

      Hi Urlop,
      don’t think so – it’s just d7 from what i’ve seen – nothing really comprable in d6