5.6 to 5.7 Migration Now Available!

Permalink 9 users found helpful
I wanted to let everyone know that we our migration add-ons for 5.6 and 5.7 are now available in GitHub. They're still under active development, but they are definitely usable. In fact, we used them to migrate the 5.7 documentation site hosted on concrete5.org to the new documentation.concrete5.org version.

A full video about how to use the tools can be found on Youtube:

https://www.youtube.com/watch?v=h9lzs2C88-s...

I blogged a bit about how the tools work as well:

http://andrewembler.com/2015/11/upgrade-concrete5-sites-57-using-ou...

As I mentioned, they're still under active development, and we'd love to accept pull requests under both the legacy and the 5.7 add-ons. I'll be adding issues to the various GitHub repositories.

http://github.com/concrete5/addon_migration_tool_legacy... (5.6 version - exports content only)
http://github.com/concrete5/addon_migration_tool... (5.7 version. imports and exports content)

Let me know what you think!

andrew
 
wagdi replied on at Permalink Reply
wagdi
Well, this is definitely exciting… can't wait to see the finished product. You guys are doing some amazing work. :)
WillemAnchor replied on at Permalink Reply
WillemAnchor
That is great news ! And the new docs section live too :D
MrKDilkington replied on at Permalink Reply
MrKDilkington
This is excellent news.

By the time 5.6 support ends, I imagine the migration tool will be even more robust.
MrKDilkington replied on at Permalink Reply
MrKDilkington
For anyone who didn't read the blog post. The migration tool replaces the Sample Content Generator.

"The export tools built into the 5.7 migration tool also replace the Sample Content Generator, which add-on and theme developers can use to generate sample content. Unlike the Sample Content Generator add-on, the migration tool can build XML exports of just portions of your site, making it much easier to work with than the old add-on."
WebcentricLtd replied on at Permalink Reply
Just watched the video. This looks great and very intelligently thought out. Can't wait to get the opportunity to try it out.
juddc replied on at Permalink Reply
juddc
Sweet! Looks really well thought out. Can't wait to give it a try.
buurvrouw replied on at Permalink Reply
buurvrouw
Great news! And it looks very promising. You're all fantastic to create such good stuff!

When does 5.7.5.3 comes available for download? I can't wait to try a migration process, but the add-on asks for version 5.7.5.3 to be installed.
PineCreativeLabs replied on at Permalink Reply
PineCreativeLabs
Question! Can this be used to migrate addons, such as core commerce?
Does this just migrate the core blocks and content, or how much support is there for migrating addons?
Mnkras replied on at Permalink Reply
Mnkras
It is independent of addons. This will NOT port addons to work with c5.7+, this is explicitly for transferring content.
buurvrouw replied on at Permalink Reply
buurvrouw
This tool is working great! Thanks so much for making this possible.

I do have a suggestion to maybe make it even more great. I bumped into a problem on a site that used 'content designer' blocks.

This add-on doesn't exist for 5.7, so i use 'block designer' to recreate the blocks. But 'block designer can't handle underscores and numbers in field handles, where 'content designer' has underscores in the field handles by default. It creates field handles like 'field_1_textbox_text'.

Is there a way you could create some kind of option in the migration process to port field handles within blocks?

I think 'designer content' is used on a great deal of 5.6 sites. So i guess a lot of people will struggle with those content blocks.
buurvrouw replied on at Permalink Reply
buurvrouw
I've bumped into another issue, which is a bit more annoying;
All imported pages get the import date as the pages creation date. For a site which relates it's pages to the date, like a blog or a portfolio this is very bad news.

Would be very handy if we could keep the original page date on import. I hope you can find a solution on this?
andrew replied on at Permalink Reply
andrew
Double-check that the export routine is putting a "public-date" node on the <page> nodes that are being exported. It should be, but it's possible that this wasn't added to the 5.6 version of the add-on. It should be

<page public-date="<?=$page->getCollectionDatePublic()?>">

in the export. If that node is there then the page will be imported with its old public date (which is the date you can change through the UI.)
buurvrouw replied on at Permalink Reply
buurvrouw
I don't think this is added in the 5.6 version. Can you tell me where to add it?
Thanks!
juddc replied on at Permalink Reply
juddc
Gave it a shot today - I get to the Download XML screen and nothing is clickable from there. Just a frozen screen. Any thoughts? Same thing on a couple of diff browsers.
katalysis replied on at Permalink Reply
katalysis
There's an issue with a fix for this on github:

https://github.com/concrete5/addon_migration_tool_legacy/issues/2...

I've tried it and it works for me.
juddc replied on at Permalink Reply
juddc
Can't find that string of code in the page.php file on the site I'm testing it on.

EDITED - Found in Updates.
PhilYoung replied on at Permalink Reply
PhilYoung
I have been using the migration tool from a 5.6.3.4 site to a clean 5.7.5.6 instal (on wampserver) and I keep getting on import
"An exception occurred while executing 'insert into Pages (cID, ptID, cParentID, uID, cInheritPermissionsFrom, cOverrideTemplatePermissions, cInheritPermissionsFromCID, cDisplayOrder, pkgID) values (?, ?, ?, ?, ?, ?, ?, ?, ?)' with params ["209", "6", null, 1, "TEMPLATE", null, "153", 0, 0]: SQLSTATE[23000]: Integrity constraint violation: 1048 Column 'cParentID' cannot be null"
Any quick thoughts on the cause of this error? Plus on looking at the db a lot of tables seem to be missing after the migration tables..odd
Dantourin replied on at Permalink Reply
Dantourin
Andrew, thanks so much for investing in such a useful tool!

Unfortunately, this is not working for me and my issue is not appearing in any of the other comments. When I click on "Add Batch", I get a file not found. Nothing is appearing in the C5 logs or my server logs.

When I try to access add_pages.php directly, I get a stripped down page, but it is accessible.

Any ideas? Thanks in advance,

Chris
PatrickHenry replied on at Permalink Reply
PatrickHenry
Andrew, you are the man (again and again).
Thanks for working on this, it's so crucial for many of us to make the leap to c5.7 efficiently.
That being said, I tried this out in c5.6.3.4 exporting a ProBlog area w/ posts dating back to 2013, more than 100 posts total I'd guess.
I build the batch to export, selecting by the blog's parent page and nothing else (no keywords, date, or other filtering). The batch creates fine, but on export I get the dreaded PHP white page of death w/ this error:
Fatal error: Call to a member function getFileID() on a non-object in /home/primary/public_html/updates/concrete5.6.3.4/concrete/core/models/attribute/types/image_file.php on line 30.

Any ideas? I posted this to the GitHub repo as an issue as well, just FYI.
Anyways, I'm more than happy to help however possible in making this thing do it's magic.
Thanks for all your efforts!
Responsive replied on at Permalink Reply
Responsive
Hi
did you manage to resolve the problem for below ?

Call to a member function getFileID() on a non-object in /home/primary/public_html/updates/concrete5.6.3.4/concrete/core/models/attribute/types/image_file.php on line 30.
PatrickHenry replied on at Permalink Reply
PatrickHenry
I haven't tried any of the updated versions since I cited this. I did cite the issue on GitHub and Andrew responded it was resolved. I recommend pulling the latest version from GitHub, which Andrew list the links for in the initial post above.
If it is resolved please let me know, time-wise I just haven't had a chance to check back on it.
Hope this helps!
Responsive replied on at Permalink Reply
Responsive
yep using latest core fromhttps://github.com/concrete5/addon_migration_tool_legacy... and installed into 5.6.3.

performed the comment out of lines in concrete/core/models/page.php as suggested inhttps://github.com/concrete5/addon_migration_tool_legacy/issues/2... and this seemed to prevent the sudden stop when exporting the batch.

Now get the error as you mentioned , but am narrowing it down to some pages so will hopefully find the cause.
buurvrouw replied on at Permalink Reply
buurvrouw
No files are added to the export batch on 5.7

I used this the legacy version af the add-on to export from 5.6 with great results, including all the used files. However i'm now trying to use the 5.7 version to export from 5.7. Also working great, except i do not get the used files, just an xml file as a result.

I've tried on different sites and different servers, but all with the same result: no files in the export batch.
Is this a bug in the latest version, or am i missing something in the export process?
buurvrouw replied on at Permalink Reply
buurvrouw
Can anyone export files with the 5.7 tool?
I do get my pages but no files are added on export. Is there someone wanting to assist me on getting this to work? I really need my files to be exported along with the pages.
ecoulter replied on at Permalink Reply
I am having a very hard time with these tools. On 5.6, if I select more than one page in a batch it seems to timeout quickly and just gives me a blank page. If I select a single page that has anything "complicated" like an image, it seems to timeout quickly and just gives me a blank page. If I have a page that contains an attribute that references an image, I sometimes get to the download page, but I am not able to download the files. I can download the xml. The download page loads with no styling and seemingly no functionality, besides go back to the batch or download xml.

What have I don't wrong? Is there a web server setting that I need to adjust? If anyone can help me, I'd be thankful.
mesuva replied on at Permalink Reply
mesuva
Have you tried the 'fix' suggested in this github issue thread?
https://github.com/concrete5/addon_migration_tool_legacy/issues/2#is...

It's a case of commenting the lines our mentioned in the 5.6 install.

I think that step might actually be necessary for all exports, not just problematic ones.
ecoulter replied on at Permalink Reply
I looked for those lines in the export function and they don't exist in my version of the file.
ecoulter replied on at Permalink Reply
ok scratch that. I was looking in the path listed, but forgot about the upgrade folders. Found those lines in /updates/concrete5.6.3.4/concrete/core/models/page.php.

That fixed the whole problem.

Thanks for the help!
buurvrouw replied on at Permalink Reply
buurvrouw
Trying to export some pages including the used files (using the 5.7 tool) I do get the right xml for the page export, but no files are added, it says 'No referenced files found.'

But this is some of the generated xml code, there are files listed:

<data table="btImageSliderEntries">
<record>
<id>
<![CDATA[ 671 ]]>
</id>
<cID>
<![CDATA[ 0 ]]>
</cID>
<fID>{ccm:export:file:ta32-w500.jpg}</fID>
<linkURL>
<![CDATA[ ]]>
</linkURL>
<internalLinkCID>
<![CDATA[ 0 ]]>
</internalLinkCID>


Can someone help me out?
indypilotcfi replied on at Permalink Reply
Hi Everyone. Is there a way to import the "Public Date" on pages? Someone asked this above but I didn't really see an answer on how to do it. I have to import a blog from 5.6 to 5.7 with a lot of posts which are sorted by the public date.

Any help?
andrew replied on at Permalink Reply
andrew
Yes, this will be a part of the next legacy 5.6 release, but in the meantime you can add public date support to export by modifying the code of your existing 5.6 site. You'll want to add it to the export() method in concrete/core/models/page.php. Find the method that looks like this:

public function export($pageNode) {
      $p = $pageNode->addChild('page');
      $p->addAttribute('name', Loader::helper('text')->entities($this->getCollectionName()));
      $p->addAttribute('path', $this->getCollectionPath());
      $p->addAttribute('filename', $this->getCollectionFilename());
      $p->addAttribute('pagetype', $this->getCollectionTypeHandle());


and add an attribute for public-date like this:

public function export($pageNode) {
      $p = $pageNode->addChild('page');
      $p->addAttribute('name', Loader::helper('text')->entities($this->getCollectionName()));
      $p->addAttribute('path', $this->getCollectionPath());
      $p->addAttribute('filename', $this->getCollectionFilename());
      $p->addAttribute('public-date', $this->getCollectionDatePublic());
      $p->addAttribute('pagetype', $this->getCollectionTypeHandle());
indypilotcfi replied on at Permalink Reply
Awesome! Thanks! I'll give this a shot tomorrow.
PixelFields replied on at Permalink Reply
PixelFields
This worked perfectly for me when I encountered the public date issue while migrating ProBlog - thank you!
indypilotcfi replied on at Permalink Reply
I hope someone can help. For some weird reason, I can't empty the trash on my instance of C5 after installing the migration tools. This is the error I'm getting:

[09-Jul-2016 20:51:25 CST6CDT] PHP Fatal error: Cannot use object of type Concrete\Core\Foundation\Processor\QueueAction as array in /home/penteraadmin/public_html/build/concrete/tools/dashboard/sitemap_delete_forever.php on line 18

Any ideas?
tzvetan replied on at Permalink Reply
What is the way to install the migration tool in the Dashboard?
Can someone please provide a link to the instructions?
Thanks!
tzvetan replied on at Permalink Reply
Here is a fully detailed way to install the addon_migration_tool_legacy in Concrete5 5.6:

• download the .zip file fromhttps://github.com/concrete5/addon_migration_tool_legacy...
• unzip the file
• copy the migration_tool folder
• paste the folder in root/pakages (root is your root folder)
• open browser
• go to your site dashboard (for example local.mysite.com/dashboard)
• login as administrator
• dashboard - extend concrete5 - the migration tool should be shown here - Install - now the migration tool should be visible inside dashboard under "Migration"
michaelfm replied on at Permalink Reply
michaelfm
I just stumpled upon this content migration tool.
After setting it up I can not access it via the dashboard. I get the following error:
error->add(t('Invalid Batch')); } if (!$this->token->validate("update_batch")) { $this->error->add($this->token->getErrorMessage()); } if (!$this->error->has()) { switch($_POST['action']) { case 'delete': $batch->delete(); $this->redirect('/dashboard/migration/batches', 'batch_deleted'); break; } } $this->view_batch($_POST['id']); } public function remove_from_batch() { $id = $_POST['id']; if ($id) { $batch = MigrationBatch::getByID($id); } if (!is_object($batch)) { $this->error->add(t('Invalid Batch')); } if (!$this->token->validate("remove_from_batch")) { $this->error->add($this->token->getErrorMessage()); } $r = new stdClass; if (!$this->error->has()) { $r->error = false; $r->pages = array(); foreach((array) $_POST['batchPageID'] as $cID) { $r->pages[] = $cID; $batch->removePageID($cID); } } else { $r->error = true; $r->messages = $this->error->getList(); } print Loader::helper('json')->encode($r); exit; } public function view_batch($id = null) { if ($id) { $batch = MigrationBatch::getByID($id); } if (is_object($batch)) { $this->set('batch', $batch); $this->set('pages', $batch->getPages()); } } public function batch_deleted() { $this->set('message', t('Batch deleted.')); $this->view(); } public function view() { $batches = MigrationBatch::getList(); $this->set('batches', $batches); } public function submit() { if ($this->token->validate("submit")) { $batch = MigrationBatch::create($_POST['description']); $this->redirect('/dashboard/migration/batches', 'view_batch', $batch->getID()); } else { $this->error->add($this->token->getErrorMessage()); } } }


Any suggestions on this one?
Mainio replied on at Permalink Reply
Mainio
michaelfm replied on at Permalink Reply
michaelfm
Thanks a bunch Mainio!
dgreer replied on at Permalink Reply
I am trying to figure out if my migration strategy is sound. I have a number of custom blocks and custom themes for each block for my site. Am I correct to assume that if I:

1) rebuild the custom blocks in version 8.0 with the same names and the same underlying templates
2) rebuild my theme with the same page types and css id/class names

that I will then be able to use the migration tool without any problems? Am I missing anything else I should be doing before trying the migration tool?

Will the migration tool move the pretty urls for underscores in the name along with other page attributes?

Any other issues to consider with custom blocks?

Thank you.
katalysis replied on at Permalink Reply
katalysis
Wrong on point 1 and right on 2...

The migration tool won't touch your custom blocks - you will need to rebuild these and migrate them manually.

You will need to rebuild your theme. The Migration Tool will then allow you to map the standard concrete5 blocks from the old site into equivalent Page Types and Areas in the new one.

The process can be a bit messy and the migration tool will only get you so far depending on the complexity of your site.
BreakfastStudio replied on at Permalink Reply
BreakfastStudio
@andrew first of all thank you very much for this Migration Tool. It helped me a transfer around 50 pages or so.

EDIT

I can see that the migration tool for 5.6 version doesn't support exporting Users. Does anyone here know how to export Users from 5.6 and import in 5.7 manually especially with Password salts involved?
devprojects replied on at Permalink Reply
I am getting this error if i want to export the batch :

t('Please enter a valid Youtube URL.')); } function __construct($obj = null) { parent::__construct($obj); if(!$this->display) $this->display='shareLong'; } function view(){ $this->set('addThisUsername', $this->username); $this->set('addThisDisplay', $this->display); } function save($data) { $args['username'] = isset($data['username']) ? trim($data['username']) : ''; $args['display'] = isset($data['display']) ? trim($data['display']) : ''; parent::save($args); } } ?>
Fatal error: Class 'AddthisBlockController' not found in concrete5.6.3.5_remote_updater\concrete\core\libraries\loader.php on line 437

What can i do ?
ppisoban replied on at Permalink Reply 1 Attachment
ppisoban
Hi Guys, Since the announcement that 5.6 is coming to end of life, I thought I'd better start to look at this.
I'm trying to upgrade from 5.6.3.4 to 5.8.4.2. I've watched the excellent video and read through this page to see what pitfalls there might be aloing the way but I seem to be falling at the first hurdle!
For some reason, 5.6.3.4 isn't spitting out a valid zip file of the images that I'm trying to add to the batch. See attached screenshot.
Has anyone else seen this?
I have tried several times now, setting up different batches each time but with the same result. The xml file seems to be fine when I download it, it's just the zip file that is giving me a headache!
nebuleu replied on at Permalink Reply
nebuleu
May I suggest you to upgrade to 5.6.3.5 before anything else ?
This version integrate "Lots of improvements to export functionality for use with the concrete5 version 8 migration tool", based on the changelog.
ppisoban replied on at Permalink Reply
ppisoban
Thanks for the reply nebuleu. I was trying to avoid upgrading first as this is quite a complicated ecommerce site with a lot of custom pages (single pages) that I didn't want to risk disturbing or causing problems!
I think I'll go and try the whole upgrade with a smaller site first, one that I know is already running 5.6.3.5 and see how that goes!
Appreciate you taking the time to reply :)
nebuleu replied on at Permalink Reply
nebuleu
In that case I strongly encourage you to create local copy of this complicated website. You will be able to test upgrade / export without constraints, and peace of mind.

For your .zip problem, have you check the web server error logs ? And the concrete5 site logs in the dashboard ?
ppisoban replied on at Permalink Reply
ppisoban
The logs are a good idea, thanks. Will go and take a look now!
JohntheFish replied on at Permalink Reply
JohntheFish
You should check addon availability. For example, the core team eComemrce package is not available with v8, so you would need to create a new shop with an alternate package and port your single page code to work with the v8 core and new shop.

As @nebulea suggests, update to 5.6.3.5 first. Don't risk the live system,
- create a clone of existing site on your dev system
- update to 5.6.3.5
- test
- export
- create v8 test site on your dev system
- import
- get it working
- upload to new live site
- point domain to new live site.

That way, your existing live site is never at risk. This won't be a simple process. It is essentially creating a new site with some of the old site content copied across. You won't have continuity of eCommerce customer accounts and purchase history without a fair bit of work to move such data.
ppisoban replied on at Permalink Reply
ppisoban
Hi John, thanks for the reply!

The ecomm side of the site is completely bespoke and the rest of the site only uses some basic add ons, so I'm not too worried about that.

Creating a clone of the site was going to be my next move. I'll do that, upgrade and retry the export/import process. The ecomm side of things uses a separate database with a custom area in the dashboard made up of bespoke single pages so I have no idea whether that's going to work at all! I'm quite happy tinkering as long as I'm not affecting the live site :)

Thanks guys!
wollastoni replied on at Permalink Reply
Hello all,

this Migration tool seems very useful, I downloaded it but when I try to install it, I have an error message saying it works only on 5.6.4 version and greater.
I have a 5.6.1 version, how can I do such a migration ?

I see that the best I can install on my old website is 5.6.3, they do not propose 5.6.4.

Thanks a lot for your help
Olivier
drbiskit replied on at Permalink Reply
drbiskit
You should be able to upgrade it to 5.6.4.0. Have you looked at these docs about how to run system upgrades?

https://legacy-documentation.concrete5.org/tutorials/manually-upgrad...

Obviously make sure you back up your files and your database first before running any upgrades - but, Concrete5.6 generally used to auto upgrade very smoothly without too many issues.