Antsy for 3.6 to start and need a project? Who wants to make an official importer for the new Twitter archives? Would think we’ll want to add that into the importers list. Would suggest importer auto-assign “status” or “aside” post format (or make it an option in the plugin to choose format). Who’s in? I volunteer to ux review and test. 
Tagged: importers Toggle Comment Threads | Keyboard Shortcuts
-
Jen Mylo
-
Andrew Nacin
If you have a Tumblr blog, can you help test an updated version of the importer? It uses their OAuth API, which requires you to create an application. It’s simple and the plugin walks you through it. Here’s the ZIP file to the beta version. You can report bugs on #22422.
Edit: The plugin has been released, the beta is over: http://wordpress.org/extend/plugins/tumblr-importer/
-
avcascade
Happy to help test this. Will try out the new importer now with a blog that’s got 2,000+ posts and report back on Trac.
-
Andrew Nacin
Wonderful.
-
-
MZAWeb
The zip link is 404ing
-
Andrew Nacin
@otto42 has fixed this.
-
-
dllh
I’ve fixed a few of the things reported so far. New zip available here.
-
zonetuke
Installed and received the following error:
Plugin could not be activated because it triggered a fatal error.
Parse error: syntax error, unexpected T_FUNCTION in /home/somaweb/public_html/see-jane-run.com/wp-content/plugins/tumblr-importer/tumblr-importer.php on line 456-
Andrew Nacin
I can reproduce. The plugin uses a closure, which isn’t available in PHP 5.2.
-
dllh
I don’t have 5.2 handy to repro, but I’ve committed a fix to the branch that removes the closure and uses a separate function instead. It doesn’t break anything and seems like it should resolve the issue. Haven’t tagged a new beta yet, as I’m not sure how frequently we want to do that.
-
-
-
Samuel Wood (Otto)
The new importer has been released as version 0.7. The above ZIP links should no longer work, just use the main plugin.
-
Aaron Brazell 11:05 pm on December 16, 2012 Permalink | Log in to Reply
I already was planning on doing this as a plugin, and I’ve been quiet for awhile. I can do this. But… I need to have the archives available to me, and my account doesn’t have it yet.
Jane Wells 11:26 pm on December 16, 2012 Permalink | Log in to Reply
Mine either, but I’ll see if I can wrangle one we can use.
Ryan Duff 11:26 pm on December 16, 2012 Permalink | Log in to Reply
Or as soon as someone gets and volunteers a copy of their archive. From the post it doesn’t seem there’s an api, but a set of html pages + xml, or json files (pick your poison)
Also, from what I’ve heard they files are monthly, so if you’ve been on Twitter for 4 years you’d be looking at 48 json files to handle w/ an importer.
Of course that’s all based off what I’ve read. I don’t have it yet. Just something to chew on.
Aaron Brazell 11:32 pm on December 16, 2012 Permalink | Log in to Reply
Yeah it’ll be an interesting challenge but I wanna see what the data looks like first. If they turn it on for me, I’ve got 6 years of archives which would be a good stress test too.
Andrew Nacin 11:33 pm on December 16, 2012 Permalink | Log in to Reply
Could handle the zip that they send you as-is. In fact, I imagine that would be the best approach for users.
Aaron Brazell 2:54 pm on December 17, 2012 Permalink
I feel like we need to be able to parse the HTML, CSV and JSON in any zip file if we approach it that way. From the user perspective, I think you’re right but I sure hope the CSV and HTML are decent enough.
Samuel Wood (Otto) 11:38 pm on December 16, 2012 Permalink | Log in to Reply
It’ll just be a matter of iterating the json. Simple stupid, for the most part.
Samuel Wood (Otto) 11:06 pm on December 16, 2012 Permalink | Log in to Reply
If they actually roll it out to people unchanged, then it should be fairly trivial. When I get it on my account, I’ll let you know.
Phil Erb 11:25 pm on December 16, 2012 Permalink | Log in to Reply
When archives are available to me, I’d love to help test.
Andrew Nacin 11:36 pm on December 16, 2012 Permalink | Log in to Reply
We now have an API for importers, which means we can add this to wp-admin/import.php the moment it is done.
When anyone gets access to their zip, please share it (or at least a sample so we can learn the format).
Aaron D. Campbell 12:29 am on December 17, 2012 Permalink | Log in to Reply
Looks like my account has the option. I’m requesting an archive and will post it somewhere to use.
Aaron D. Campbell 12:39 am on December 17, 2012 Permalink | Log in to Reply
Looks like it comes down to a bunch of poorly formatted CSV files. Here’s what they sent me: http://s.ran.ge/0o2F011v3I2p
The CSVs don’t quote data, so commas in the tweet will be a bit of a pain. However, since it’s ID, time, tweet you can probably just process the ID and time, then assume the rest is the tweet (trimming the strange trailing comma).
deltafactory 6:04 am on December 17, 2012 Permalink | Log in to Reply
I’d be curious to peek at the format. Your link is broken – not sure if that was intentional or not.
Aaron D. Campbell 8:56 pm on December 17, 2012 Permalink | Log in to Reply
Sorry about the broken link. I uploaded the export to the Trac ticket: http://core.trac.wordpress.org/ticket/22981
Jane Wells 6:12 pm on December 19, 2012 Permalink | Log in to Reply
I mentioned to Ev that commas in tweets was requiring some hackiness.
Pete Mall 9:23 pm on December 19, 2012 Permalink | Log in to Reply
We don’t have to worry about it since we are using the JSON instead of the CSV.
Alex Mills (Viper007Bond) 9:30 pm on December 19, 2012 Permalink
True, but still a bug. Delimiters in CSVs need to be escaped.
Myatu 6:53 am on December 17, 2012 Permalink | Log in to Reply
Wouldn’t it be more sensible to have this as a 3rd party plugin, rather than having to maintain more bloat?
Peter Westwood 10:40 am on December 17, 2012 Permalink | Log in to Reply
It will be a plugin anyway not in the core download – all the importers are plugins now.
Myatu 6:21 am on December 18, 2012 Permalink | Log in to Reply
Actually forgot about that! Shows how often I’ve used that feature. I stand corrected
Jane Wells 11:39 am on December 17, 2012 Permalink | Log in to Reply
As @westi states, all importers are plugins, not core code. I didn’t specify that in my post, since I took it as a given that core developers know that.
Simon Wheatley 8:34 am on December 17, 2012 Permalink | Log in to Reply
Note that the current Twitter IDs overflow (?) and corrupt if you convert them to integers on a 32bit system. That one has got me before. (Apologies if that’s teaching everyone to suck eggs.)
Also, would you mind putting in a filter for the post data before save… I’d prefer to store tweets in a custom post type. Ta!
Brad Williams 2:59 pm on December 17, 2012 Permalink | Log in to Reply
Was thinking the same thing regarding CPTs. A filter would be perfect
Andrew Nacin 6:49 pm on December 17, 2012 Permalink | Log in to Reply
I’ve outlined a potential plan for such an importer on a Trac ticket: http://core.trac.wordpress.org/ticket/22981. If you want to continue to discuss the idea, feel free to do so here. Implementation can occur on the ticket. (This is a plugin, but an official importer is also a core priority, hence the use of core trac.)
I’ve also uploaded a tweet archive contributed by @chadhuber to the ticket. It does contain sane json.
Pete Mall 6:56 pm on December 17, 2012 Permalink | Log in to Reply
@aaroncampbell and I worked on the plugin yesterday. There’s an early patch on the ticket already. I’m working on it today and will add an updated patch eod.
Beau Lebens 9:23 pm on December 17, 2012 Permalink | Log in to Reply
In case it helps, I already made one, packaged in here: http://wordpress.org/extend/plugins/keyring-social-importers/
Pete Mall 9:26 pm on December 17, 2012 Permalink | Log in to Reply
Looks like we should be able to use your “parser”.