Internationalization

Multibyte URL and Search will be available - NEED HELP

Permalink December 29, 2010 at 5:43 AM

First of all, all of the following work has been done by Tao.
http://www.concrete5.org/profile/-/1381/...

Since October, we've been working to release 5.4.1.1 Japanese version. And we decided to take our time to support some of multilinguaal & multibyte issues.

The following is the major achievement of our works...

- Add multi-byte URL (can use Japanese, Chinese, Korean, Arabic as URL)
- Multi-byte search (can search Japanese, Chinese, Korean, Arabic and etc.)
- Fixing bugs of Tags search
- Load new language files before conducting upgrade

We decided to

- disable full-page cache upon install
- Switch getAttributeKeyDisplayHandle() back to getAttributeKeyName()
(VERY IMPORTANT)

We're hoping to merge these functions into the core system. So that we don't have to re-patch every time... and shorten the time-lag between English and Japanese version. Moreover, other language users will be able to benefit from the changes anyway.

Tao and I were the only ones who mainly worked on this change. We sometime just commit the SVN rather than separate the commits depends on the issue...

We are not really well-documented on these changes. We have been busy paying our own bills first, and trying to finish the translation.

Therefore, I would like to take a volunteer who can take a look at our changes, and re-document and report it to Andrew.

We have not finalize the 5.4.1.1 Japanese version yet.... Please reply and PM me.

You are very welcome to take a look at our SVN.

concrete5 Japan SVN
svn[dot]concrete5-japan[dot]org/svn/c5japan/

(Replace "[dot]" to "." & We may move it to GitHub or other service in the near future)

Tao replied on Dec 29, 2010 at 8:02 am Permalink Reply

Thank you katz.

concrete5 in non-alphabetic characters used to handle or do not define a page when adding a page, the page that is registered with the database handle things as a null.

Some add-ons do not work normally because of this issue.

So I like to use for page handle URLencode also use non-alphabetic character.

An important part of the code is this.
/concrete/helpers/text.php

function sanitizeFileSystem($handle, $leaveSlashes=false) {
      $handle = trim($handle);
      $handle = str_replace(PAGE_PATH_SEPARATOR, '-', $handle);
      $searchMulti = array(
         "ä",
         "ö",
         "ß",
         "ü",
         "æ",
         "ø",
         "å",
         "é",
         "è"   
      );
      $replaceMulti = array(

Viewing 15 lines of 43 lines. View entire code block.

   function sanitizeFileSystem($handle, $leaveSlashes=false) {
      $handle = trim($handle);
      $handle = str_replace(PAGE_PATH_SEPARATOR, '-', $handle);
      $searchMulti = array(
         "ä",
         "ö",
         "ß",
         "ü",
         "æ",
         "ø",
         "å",
         "é",
         "è"   
      );
 
      $replaceMulti = array(
         'ae',
         'oe',
         'ss',
         'ue',
         'ae',
         'oe',
         'aa',
         'e',
         'e'
      );
 
      $handle = str_replace($searchMulti, $replaceMulti, $handle);
 
      $searchNormal = array("/[&]/", "/[\s|.]+/", "/-+/");
      $searchSlashes = array("/[&]/", "/[\s|.]+/", "/-+/");
      $replace = array("and", "-", "-");
 
      $search = $searchNormal;
      if ($leaveSlashes) {
         $search = $searchSlashes;
      }
 
      $handle = preg_replace($search, $replace, $handle);
      if (function_exists('mb_substr')) {
         $handle = mb_strtolower($handle, APP_CHARSET);
      } else {
         $handle = strtolower($handle);
      }
      $handle = trim($handle, '-');
      $handle = str_replace('-', PAGE_PATH_SEPARATOR, $handle);
      return $handle;
   }

/concrete/models/page.php

function getCollectionPath() {
   return $this->getEncodePath($this->cPath);
}
public static function getCollectionPathFromID($cID) {
   $path = Cache::get('page_path', $cID);
   if ($path != false) {
      $this->getEncodePath($path);
   }
   $db = Loader::db();
   $path = $db->GetOne("select cPath from PagePaths inner join CollectionVersions on (PagePaths.cID = CollectionVersions.cID and CollectionVersions.cvIsApproved = 1) where PagePaths.cID = ?", array($cID));
   $path .= '/';
   Cache::set('page_path', $cID, $path);
   return $this->getEncodePath($path);
}
function getEncodePath($path){

Viewing 15 lines of 25 lines. View entire code block.

function getCollectionPath() {
   return $this->getEncodePath($this->cPath);
}
 
public static function getCollectionPathFromID($cID) {
   $path = Cache::get('page_path', $cID);
   if ($path != false) {
      $this->getEncodePath($path);
   }
 
   $db = Loader::db();
   $path = $db->GetOne("select cPath from PagePaths inner join CollectionVersions on (PagePaths.cID = CollectionVersions.cID and CollectionVersions.cvIsApproved = 1) where PagePaths.cID = ?", array($cID));
 
   $path .= '/';
 
   Cache::set('page_path', $cID, $path);
   return $this->getEncodePath($path);
}
 
function getEncodePath($path){
    if(mb_strpos($path,"/") !== false){
      $path = explode("/",$path);
      $path = array_map("urlencode",$path);
      return implode("/",$path);
    }else if(is_null($path)){
      return NULL;
    }else{
      return urlencode($path);
    }   
}

/concrete/libraries/request.php

public function getRequestCollectionPath() {
   // I think the regexps take care of the trimming for us but just to be sure..
   $cPath = trim($this->cPath, '/');
   if ($cPath != '') {
      return '/' . urldecode($cPath);
   }
   return '';
}

public function getRequestedPage() {
   $path = $this->getRequestCollectionPath();
   $r = Cache::get('request_path_page', $path);
   if ($r == false) {
      // Get the longest path (viz most specific match) that is contained
      // within the request path
      $db = Loader::db();
      $r = $db->Execute("select cID,cPath from PagePaths where ? LIKE CONCAT(replace(cPath, '_','\_'),'%') ORDER BY LENGTH(cPath) DESC LIMIT 0,1", array($this->getRequestCollectionPath()));
      $r = $r->FetchRow();
      if (is_array($r)) {
         Cache::set('request_path_page', $path, $r);
      }         
   }   
   if (is_array($r)) {
      $req = Request::get();

Viewing 15 lines of 25 lines. View entire code block.

public function getRequestedPage() {
   $path = $this->getRequestCollectionPath();
   $r = Cache::get('request_path_page', $path);
   if ($r == false) {
      // Get the longest path (viz most specific match) that is contained
      // within the request path
      $db = Loader::db();
      $r = $db->Execute("select cID,cPath from PagePaths where ? LIKE CONCAT(replace(cPath, '_','\_'),'%') ORDER BY LENGTH(cPath) DESC LIMIT 0,1", array($this->getRequestCollectionPath()));
      $r = $r->FetchRow();
      if (is_array($r)) {
         Cache::set('request_path_page', $path, $r);
      }         
   }   
 
   if (is_array($r)) {
      $req = Request::get();
      $cPath = $r['cPath'];
      $cID = $r['cID'];
      $req->setCollectionPath($cPath);         
      $c = Page::getByID($cID);
   } else {
      $c = new Page();
      $c->loadError(COLLECTION_NOT_FOUND);
   }
   return $c;
}

Have confidence much.
In this case, the destination URL of the "$ Page-> cPath" can not be obtained.

I'm sure non-English speaking people would have a similar problem.

Help each other and like each other, reducing the extra work I hope.

Tao replied on Dec 29, 2010 at 8:09 am Permalink Reply

When you add an attribute to define why I use the attribute name in the dashboard?

I want to know the intent of the core team.

Tao replied on Jan 12, 2011 at 4:24 am Permalink Reply 1 Attachment

Attached all diff from 5.4.1.1en to 5.4.1.1.1ja. Please check it.

Forums

Internationalization

Multibyte URL and Search will be available - NEED HELP

Code

Post Reply

Delete Post

Mark Post as Spam

Destroy Spammer

Sign In?