Automated C5.4.1 setup results in Package install failure
Permalink
Gentlemen,
I thought I'd post here instead of the Installation help forum - as I really need some developer feedback on this issue. You can see the original thread here:
http://www.concrete5.org/community/forums/installation/installing-p...
What I haven't mentioned in the original thread is how my site is set up. What we have done is built an automated system that sets up a complete C5 site on our server by automating ISPConfig. Now, everything was working beautifully untill we built an automated package deployment system.
Before describing the problem - Let me walk you through the process we go through when we set up a site:
1) Set up the vhost/db/ftp and all that jazz
2) Copy C5 files to the vhost destination and symlink in a C5 core
3) Do a DB dump of our, original, virgin C5 base install and import it to our new DB
4) Set all filesystem rights and create the config
All of this has been THOROUGHLY tested and everything is a perfect mirror image of our base C5 install - except for it being on another location on the server.
Now, the problem is that if we now want to install a package on this fresh setup - it fails with the error message:
An unexpected error occurred.
not found. Please check that the block controller file contains the correct class name.
As you can see the class variable is empty for some reason (usually that error should display the classname which it cannot find, as far as I can gather)
To reiterate: Installing packages on our base C5 installation which gets duplicated in the fresh install works just fine. So does setting up a site manually, then running the C5 install and then installing a package.
After much debugging and rummaging around the code (and dark corners of my brain) - I have come to the tentative conclusion that this could most likely be a problem resulting from the fact that our automatically installed site is using a DB dump from an existing site. I.e. the DB isn't "freshly created" by the C5 installer
I base this on the fact that there is nothing wrong with the filesystem or rights - nor with my config file. Then all there is left is the data in the DB which might be affecting package installation. I have yet to find the culprit, despite browsing thoroughly through the C5 DB tables.
As mentioned in my original post - this is what I found when stepping through the package install code:
================================
The reason why package installation fails is because the method
$pkgHandle = $this->getPackageHandle();
called from function _getClass() in block_types.php (around line 555)
Fails. I.e. it results in $pkgHandle being NULL - which in turn causes the following if else file_exists statements in this function - which try to include controller.php from the package - to fail (it ends up looking for controller.php in my blocks subdir - which is wrong)
looking at getPackageHandle(); this directs me to PackageList::getHandle($this->pkgID);
Which is where I get stuck - as now we are so deep in the core, that I can't quite wrap my head around what is going on. Seems like the package I am trying to install hasn't been added to an internal list yet - and this is why the handle ends up being NULL - I could be wrong though.
================================
So, my question is - to you my dear C5 developers - is my hunch correct? That is, is there some value which gets saved in the Database at install-time which could result in package installation to fail if said database was duplicated for a site at another filesystem location on the server?
I sincerely hope my explanations haven't been too convoluted and that you "get" the problem. I look forward to your replies!
Psst
I thought I'd post here instead of the Installation help forum - as I really need some developer feedback on this issue. You can see the original thread here:
http://www.concrete5.org/community/forums/installation/installing-p...
What I haven't mentioned in the original thread is how my site is set up. What we have done is built an automated system that sets up a complete C5 site on our server by automating ISPConfig. Now, everything was working beautifully untill we built an automated package deployment system.
Before describing the problem - Let me walk you through the process we go through when we set up a site:
1) Set up the vhost/db/ftp and all that jazz
2) Copy C5 files to the vhost destination and symlink in a C5 core
3) Do a DB dump of our, original, virgin C5 base install and import it to our new DB
4) Set all filesystem rights and create the config
All of this has been THOROUGHLY tested and everything is a perfect mirror image of our base C5 install - except for it being on another location on the server.
Now, the problem is that if we now want to install a package on this fresh setup - it fails with the error message:
An unexpected error occurred.
not found. Please check that the block controller file contains the correct class name.
As you can see the class variable is empty for some reason (usually that error should display the classname which it cannot find, as far as I can gather)
To reiterate: Installing packages on our base C5 installation which gets duplicated in the fresh install works just fine. So does setting up a site manually, then running the C5 install and then installing a package.
After much debugging and rummaging around the code (and dark corners of my brain) - I have come to the tentative conclusion that this could most likely be a problem resulting from the fact that our automatically installed site is using a DB dump from an existing site. I.e. the DB isn't "freshly created" by the C5 installer
I base this on the fact that there is nothing wrong with the filesystem or rights - nor with my config file. Then all there is left is the data in the DB which might be affecting package installation. I have yet to find the culprit, despite browsing thoroughly through the C5 DB tables.
As mentioned in my original post - this is what I found when stepping through the package install code:
================================
The reason why package installation fails is because the method
$pkgHandle = $this->getPackageHandle();
called from function _getClass() in block_types.php (around line 555)
Fails. I.e. it results in $pkgHandle being NULL - which in turn causes the following if else file_exists statements in this function - which try to include controller.php from the package - to fail (it ends up looking for controller.php in my blocks subdir - which is wrong)
looking at getPackageHandle(); this directs me to PackageList::getHandle($this->pkgID);
Which is where I get stuck - as now we are so deep in the core, that I can't quite wrap my head around what is going on. Seems like the package I am trying to install hasn't been added to an internal list yet - and this is why the handle ends up being NULL - I could be wrong though.
================================
So, my question is - to you my dear C5 developers - is my hunch correct? That is, is there some value which gets saved in the Database at install-time which could result in package installation to fail if said database was duplicated for a site at another filesystem location on the server?
I sincerely hope my explanations haven't been too convoluted and that you "get" the problem. I look forward to your replies!
Psst
if you have packages that are always deployed with every new install you can simply add them to your concrete5 shared/replicated directory under ~/concrete/packages/*
Those get installed automagically.
Those get installed automagically.
Hello lads,
Thanks so much for your replies. I'vet been so fortunate to have been on a 2 week vacation to Iceland - so I haven't had time to look at this again untill now. In any case, lemme run through your suggestions:
Remo - Paths: I am fairly sure my paths are AOK and that the packages are placed in the correct location, as I otherwise would not be seeing them in the back-end overview.
Remo - Database: You do mention that there is a list maintained in the Database. As far as I can see, then only list which is in the Database is the list of installed Packages. My hunch, as described above, was that there was something in the database which - when the site was running at another location on the server than the one originally installed on - prevented packages from being installed.
Remo - Cache: I tried clearing the cache in the "source" virgin site without any effect on this problem.
Scott: Unfortunately your suggestion is not something applicable to our project as we, as mentioned, use a symlinked core C5 directory which is shared across all our auto-setup sites. I will keep your, otherwise excellent, tip in mind for future projects however.
In any case - I am still stuck. What I cannot quite fathom is where to look next. That is, as far as I can see - as mentioned in my original post - the my auto-setup sites are pretty much indistinguishable from the original site, with the exception that they are owned by another web user, and are placed on a different location on the server.
So I keep circling back to this being some sort of weird permission problem - but the pieces of the puzzle don't quite add up.
So - Here's hoping any of you guys can think of something further in my environment that I could check...
Thanks for readin'!
Psst
Thanks so much for your replies. I'vet been so fortunate to have been on a 2 week vacation to Iceland - so I haven't had time to look at this again untill now. In any case, lemme run through your suggestions:
Remo - Paths: I am fairly sure my paths are AOK and that the packages are placed in the correct location, as I otherwise would not be seeing them in the back-end overview.
Remo - Database: You do mention that there is a list maintained in the Database. As far as I can see, then only list which is in the Database is the list of installed Packages. My hunch, as described above, was that there was something in the database which - when the site was running at another location on the server than the one originally installed on - prevented packages from being installed.
Remo - Cache: I tried clearing the cache in the "source" virgin site without any effect on this problem.
Scott: Unfortunately your suggestion is not something applicable to our project as we, as mentioned, use a symlinked core C5 directory which is shared across all our auto-setup sites. I will keep your, otherwise excellent, tip in mind for future projects however.
In any case - I am still stuck. What I cannot quite fathom is where to look next. That is, as far as I can see - as mentioned in my original post - the my auto-setup sites are pretty much indistinguishable from the original site, with the exception that they are owned by another web user, and are placed on a different location on the server.
So I keep circling back to this being some sort of weird permission problem - but the pieces of the puzzle don't quite add up.
So - Here's hoping any of you guys can think of something further in my environment that I could check...
Thanks for readin'!
Psst
Alright, I found the solution after much debugging and tracing through the Concrete5 Code.
The problem ended up with being with my cache. The problem was that when we duplicate the original site on the other location on the server, we updated all the files to have the appropriate rights - that is, all files got the new web user/group rights.
Problem was that everything in the /files/ subdirectory (including cache) has to be owned by www-data (not the web root user)
So by doing a further update in our setup code which sets www-data:www-data as the owner/group of everything in the files subdirectory - we fixed the problem!
Thanks for your feedback and for listening!
Psst
The problem ended up with being with my cache. The problem was that when we duplicate the original site on the other location on the server, we updated all the files to have the appropriate rights - that is, all files got the new web user/group rights.
Problem was that everything in the /files/ subdirectory (including cache) has to be owned by www-data (not the web root user)
So by doing a further update in our setup code which sets www-data:www-data as the owner/group of everything in the files subdirectory - we fixed the problem!
Thanks for your feedback and for listening!
Psst
psst -
Our web company ran into a very similar problem to the one you've just apparently solved. Although, your solution isn't perfectly clear to me.
After rummaging through the code, same as you did, we too had a NULL class variable and a NULL $pkgHandle variable, etc.
First of all, is 755 - www-data user and 777 - web root user? just wanted to check those references.
We first tried setting all files in the whole install to 777, except that the files\cache folder, when set to 777 the website will not load and instead says "500 internal server error." So, we set that to 755.
With these setting we still ran into the "not found. Please check the block controller...." error.
You said "When we set the files subdirectory to 755 (www-data)
Problem was that everything in the /files/ subdirectory (including cache) has to be owned by www-data (not the web root user)"
Here it seems you are saying that you set the \files subdirectory to 755. Yes?
Then you said "So by doing a further update in our setup code which sets www-data:www-data as the owner/group of everything in the files subdirectory - we fixed the problem! "
You mention "setup code" - what is that?
When you say you set "www-data:www-data as the owner/group", are you saying that you set the files\ subdirectory to 755? What's the difference between
A) setting the files subdirectory to 755, including every file in that subdirectory,
and
B) what you did
Thanks! Sorry this is so detailed - We gotta get to the bottom of this :)
Our web company ran into a very similar problem to the one you've just apparently solved. Although, your solution isn't perfectly clear to me.
After rummaging through the code, same as you did, we too had a NULL class variable and a NULL $pkgHandle variable, etc.
First of all, is 755 - www-data user and 777 - web root user? just wanted to check those references.
We first tried setting all files in the whole install to 777, except that the files\cache folder, when set to 777 the website will not load and instead says "500 internal server error." So, we set that to 755.
With these setting we still ran into the "not found. Please check the block controller...." error.
You said "When we set the files subdirectory to 755 (www-data)
Problem was that everything in the /files/ subdirectory (including cache) has to be owned by www-data (not the web root user)"
Here it seems you are saying that you set the \files subdirectory to 755. Yes?
Then you said "So by doing a further update in our setup code which sets www-data:www-data as the owner/group of everything in the files subdirectory - we fixed the problem! "
You mention "setup code" - what is that?
When you say you set "www-data:www-data as the owner/group", are you saying that you set the files\ subdirectory to 755? What's the difference between
A) setting the files subdirectory to 755, including every file in that subdirectory,
and
B) what you did
Thanks! Sorry this is so detailed - We gotta get to the bottom of this :)
Hi,
No problem. I think your confusion stems from the fact that you don't know that rights and ownership are two different things.
You need to set ownership of the cache directory and files to www-data:www-data - You do this by issuing the following command in a shell:
chown -R www-data:www-data /var/www/whatever.com/web/cache/*
Where you'd on the other hand use the command "chmod" to change rights.
I hope that makes sense. You cannot do this operation via. an FTP client.
And what I meant with our "setup code" is that I was referring to our own system that automatically sets up a concrete5 installation for us (hence the wrong ownership of the cache folder in the first place).
Don't hesitate to ask if you need more detail ...
No problem. I think your confusion stems from the fact that you don't know that rights and ownership are two different things.
You need to set ownership of the cache directory and files to www-data:www-data - You do this by issuing the following command in a shell:
chown -R www-data:www-data /var/www/whatever.com/web/cache/*
Where you'd on the other hand use the command "chmod" to change rights.
I hope that makes sense. You cannot do this operation via. an FTP client.
And what I meant with our "setup code" is that I was referring to our own system that automatically sets up a concrete5 installation for us (hence the wrong ownership of the cache folder in the first place).
Don't hesitate to ask if you need more detail ...
whoops... The command should be (err, I think):
chown -R www-data:www-data /yourfulppathtoc5install/files/*
chown -R www-data:www-data /yourfulppathtoc5install/files/*
We fixed it another way actually. We inherited the website from another company and they had some strange cache config settings. We ended up deleting libraries\cache.php and took a few lines out of the config file site.php .. that was to allow us to set the files\cache folder to 777 without getting that error #500 mentioned above.
However, you at least got us looking in the right direction. Thanks for all of your help!
However, you at least got us looking in the right direction. Thanks for all of your help!
Awesome :)
There's no internal list beside the database, hard to see the problem if your database is up to date..
Also, did you clear the cache before you exported the c5 files? I often had problem when I kept the old files in the cache directory.