github... all joking aside. I honestly just think they are really backed up with 5.5. I could be wrong but that's my personal impression.

You could be right. It's just that my frustration stems from noticing that other add-ons are being approved, while mine are not.

I have a couple myself.

It's nothing personal I assure you. There is just a TREMENDOUS amount of work going into v5.5. It's consuming a ton of resources at the moment.

Be patient. They'll get through.

ChadStrat

you have no idea. theres around 45 pending items right now, and the c5 team is swamped with 5.5, so do't get upset that other things are getting approved any yours aren't, theres a lot of addons in the same boat.

Well that's encouraging then that there's others with stuff awaiting approval. Around 45 items? Wow... and to think I'm about to submit another item tonight or tomorrow!

Personally, I just really wish the c5 team would communicate this somehow, so we had some idea of what's going on.

The list awaiting approval is overwhelming and growing while lots of work is going into 5.5. Nevertheless, Growthcurve is expressing a sentiment that has been lurking behind a number of addon submissions and maybe just a little bit more official communication on this would help keep everyone happy.

Hi, I understand your fustration, The core team said in the last totally ramdom i think that they are focusing on last c5 version, as well this his the worst period to submitions as its christmas period and I would guess that some people inculding myself as new on prb are actually taking some time off after a year of hard work.
I think that althought the list his quite big, it should all be starting to be approved quite soon.
Paulo

Same frustration here, also 3 add-ons have been almost a month in the PRB. Only one comment in one of them from week ago stating that it works and installs as it should.

I haven't complained before because I kinda thought that this has something to do with the busy 5.5 release. However, to me it doesn't seem like a "free market" if time to market is a month and there is one "dictator" who can approve their own/partners' add-ons immediately. (Don't get me wrong here, it might seem like harsh point but I'm not that serious)

frz, heads up here, maybe it's time hire someone new to your growing business? Everything cannot be "crowdsourced" at least as long as you keep the final "approval" button in-house.

Antti

I completely understand how frustrating it is to have a barrier in front of you after putting so much work into building something. I certainly wish things could move faster here too. To be frank, there hasn't been any official "don't worry about the PRB" discussion at the office, so there's nothing I would have posted to the blog about it. Totally Random is a good spot to put us on the hook for an answer as it's live, and yes clearly we're very focused on 5.5 now, but ya know we always have 1000 things to worry about. I do appreciate being let off the hook by several people in this thread with 5.5 as an excuse, but I can't honestly say there won't be all sorts of other things we'll be worried about in 2012.

I also know you all agree that our marketplace is very different than other open source projects. It's not a "free market" or a code repository. It's a safe place for anyone to shop and extend their website knowing that installing something won't break their site. That is a huge differentiator from projects like drupal, where even the strongest advocates will admit they basically have /their/ copy of drupal with all the modules they know and love - and if you don't you're SOL.

We intended to have a safe marketplace from day one, we knew it would be hard to do, or everyone else would do it this way. So I just want to reiterate that. With all respect for everyone's tremendous work and contribution - this is not a free market. It's my store and I'll be as careful as I can about what I sell to our customers. That's a good thing, you want me to do that, it will help everyone's eventual sales in the end. This difference in approach is a huge part of the reason why that 2011 Open Source Market Share Report says we're growing by leaps and bounds faster than every other open source cms. The highly curated nature of our marketplace should be WHY you want to sell stuff in it.

I certainly completely agree that many business practices can not be "crowd sourced." I completely disagree that this is one of those things. This is basic testing guys. Very frequently when I do send one of my employees through the submissions they're posting things like "This doesn't install for me." Is it really necessary for me to spend some of the valuable revenue the marketplace generates paying people to test if 3rd party developers have read the package format instructions? Or if their thumbnail is the wrong shape? I hope not. Don't get me wrong, I will eventually if I have to, but I think it is reasonable that I do so at my own financial pace. ;)

Case in point, I looked up Mainio's three add-ons. Of the three, only one of them has one comment at all. Yes, it's from one of my employees. Admittedly my employee is saying it looks good and there's probably a discussion we should have at my office about updating our official approval policy/process when at least one employee here says yes, but seriously? It's called the PEER review board. I see plenty of threads that have many many comments from Matt, Greg or Andrew (all on my payroll) and the add-on still doesn't work. I just approved a dozen new members to the PRB, which numbers around 30 people now. If Mainilo's add-ons all had 2 or 3 different people saying "yeah this works great!" it'd be a lot easier for me to feel comfortable having Matt and Greg approve more stuff without having Andy look at it as well. That's the simple truth from my prospective. The testing the PRB does is tremendously valuable to us.

It's not a lack of staff on my part. We're hiring the right amount of resources to solve plenty of our challenges. Matt and Greg have done an awesome job becoming expert support guys in 2011, and their feedback in the PRB has clearly been invaluable to many. We've got 4 more support interns starting in the new year. The marketplace is an important revenue stream for us, but it has to cover an awful lot of different things. I'm not going to put someone in charge of it who is new to concrete5. The speed of its growth is important, but not the most important thing on my radar.

Yes. I recognize we approved our own theme yesterday, along with a handful of other add-ons that looked good. We don't have any "partners". The ones we approved we approved because Greg had the time and ability to easily test them and felt confident about it. Andy didn't have a chance to look. I trusted Greg, and I had Matt and Greg double check each other's work on Holiday Yogurt. It also helped that someone else had commented that it worked okay for them.

Many of the people in this discussion are PRB members. Be careful pointing fingers gang, you may be pointing one at me, but you're pointing three back at your self. If each one of these items had several posts from PRB members saying it works perfectly, I'd happily pay my boys to do a final check and approve everything first thing monday. Yes, I get that it's a volunteer board, but no one was drafted. ;) I agree it'd be great if things we're approved faster. I also get that I'm playing both sides by saying "it's not a free market" but making the approval process "community driven," but the fact remains - that's what it is. Lets turn some of this frustration energy into testing energy so it's easier for me to pay the people to do what you want. No? Just sayin'.

It'd be awesome to see some PRB members in here chiming in with "Let me test your stuff right now Mainio, that will make it easy for the core team to stay focused on the stuff we want them working on, while still keeping this process moving."

So there's more story than you asked for. ;)
We will set aside some time to do a complete pass on the submissions next week. We will be cleaning out old stuff that didn't work and hasn't had any activity in 30 days. We will be eagerly trying to approve things, so if the PRB could throw any energy they do have at helping doing that initial testing - that'd really help.

Franz. Do you apply to get on the PRB, or do you guys select people for it. I'd like to get more involved here, but no idea what the process or entry requirements are?

there was a thread a while back, just pm frz.

Thanks Jack - will do

[rant]
The issue I see with this is why. The core team has every reason to want to see addons approved. More paid addons=more revenue and more free addons=more people interested in c5. you could argue that the prb guys benefit too, because it means more people using c5, which means more potential clients. However this is an indirect relationship, and it can be hard to put in a lot of work now with the promise of work in the future. SO

there needs to be more rewards for participating in the prb and offering insightful comments. One way to do that is power. Everyone likes power :) But allowing some way for enough prb guys to say yeah, this works, and not have to have one of the core team to approve it. I realize that this means extra coding, but I think allowing the prb to actually have a voice in what gets approved or not would be great. If you guys test everything yourself, then you test things regardless of whether or not prb people say it works, so saying this works for me doesn't actually help to speed up the process.

Another way to encourage reviewing this is to grant licenses. Yeah, you can get free downloads of everything, but I still buy licenses of things i've reviewed. Making it official I think would remove the gray area there.

So, you can point three fingers at us, but make sure you've got a leg to stand on. While I do want c5 to grow, and I think the rest of the prb does too, you guys are the ones that benefit most from this stuff. If the mp is even 25% of your revenue, then you guys should be spending more time on it, and not letting the list grow and grow. Admittedly there are still things that have issues in it, but cmon, if the prb can't press the green ok button, then how is it helping you. It can point out errors, but at a certain point there needs to be more activity from the core team to approve submissions that people say are good to go, or no ones commented on.
[/rant]

"More paid addons=more revenue and more free addons=more people interested in c5"

The west coast kids are all about "curation" these days. It's the hip thing, don't take it personally. It's just kind of trendy right now, not necessarily the most logical.

Uhhhh what? my whole point was that the curation should be going on faster, I wasn't arguing that there shouldn't be curation, just that it should be faster.

This topic strikes a chord with me, so I'd like to share some thoughts. Franz, I understand your desire to ensure a favorable experience for end users with regard to their marketplace purchases. However, I wonder if the review and approval system currently in place is really the best approach for everyone involved (end users, developers, and C5 alike). Following are some problems I see.

Approval Backlog
--------------
That's what prompted this thread, so I won't spend time on it other than to say that my experience with the review process has been...well...less than optimal (and neither an approaching holiday nor a pending major C5 release were contributing factors).

No Incentive to Review
-------------------
I'm not on the PRB for a few reasons, not the least of which is that I simply don't have time to devote to testing someone else's software. In fact, I find myself wondering what would motivate anyone to be a part of the review process - especially other developers.

Obviously, anyone using or developing for C5 wants it to be successful; but really, what does a developer gain from spending time examining another developer's software? If anything, I see potential "conflicts of interest" arising.

Similarly, what developer who makes the substantial investment learning the C5 ropes, creating added value for the CMS, and documenting their efforts with clear, concise, copiously commented code (hey, is that what C5 stands for?) wants to simply hand their source code over to "just anyone" (as opposed to paying customers)?

Support Quality Not Guaranteed
---------------------------
Since the approval process is a one-time thing, a theme or add-on can pass the bare minimum approval threshold and then suffer from poor developer support or not be maintained or updated and left to languish, thereby creating an unpleasant experience for end users - precisely what the approval process was created to avoid! The current system doesn't address the "quality of support" issue, and that's a very important and relevant issue for most people.

Suggestions
----------
What if there were two broad classifications of packages in the marketplace - those vetted by C5 and those not (perhaps indicated by a special "badge" or other method of clearly making the distinction). And what if the peer review process was optional.

From the developer's perspective, they would be able to decide for themselves on a case-by-case basis if a particular package of theirs warranted the additional scrutiny. A more experienced developer with a good track record might decide to forgo the review process for a simple package, knowing that their item will appear in the marketplace without the "official C5 seal of approval".

Alternatively, the developer might opt for the peer review process and perhaps offer a free license to those who provide useful feedback. That might incentivize reviewers to participate and provide high quality comments and criticism.

OR perhaps the ability to opt out of the review process is "earned" by developers based upon the number of add-ons they have in the marketplace along with their support record.

From C5's perspective, they would still get their slice of the revenue pie, the review backlog would be reduced, and fewer resources would be wasted.

From the end user's perspective, they would know exactly what they're purchasing. For non-vetted submissions, they have the add-on description, reviews, as well as the developer response and support record upon which to base their purchase decision. I think as long as the distinction between vetted and non-vetted items is made clear and obvious, there is no reason to suspect that users would be any less happy with their purchases.

Thoughts, comments?

-Steve

If anyone wants to offer addons that haven't gone through the PRB, they can. They just can't do it through concrete5.org. Having looked at the muddled and confused state of the Drupal And Joomla equivalents of the C5 marketplace, the quality of the C5 marketplace is a great strength that cant be discarded.

I think the current griping has grown from addon developers shooting in the dark. Normally this hasn't been a problem because review or approval of addons that worked was fairly fast. At the moment there is a slowing for 5.5 and also a change of rules while addons are waiting (most in PRB were developed for 5.4.x, but are now being evaluated against 5.5, so there is an high proportion of returns).

I think communication could be clearer. For example,http://www.concrete5.org/developers/marketplace-submission-rules/... does not make it clear that addons will only be approved if they work in 5.5. So new addons are continuing to arrive in the PRB that don't meet that requirement.

At the same time, the github download of 5.5 has short php tags, so just getting a development copy of 5.5 up and running is not as simple as it could be. A standard XAMP environment cant run it without changing php.ini. Its only one line in the file, but it is another hoop to jump through that doesn't need to be there.

The great strength of C5 is the simplicity and user friendliness of creating and editing a site. This thread really arises because some basic processes are straying from that principle.

The bottleneck isn't one sided. Having had several of my own addons go through the PRB quickly with only minor comments, now on the PRB I am surprised by the number waiting that have major issues. The change of rules to 5.5 may be part of it, but I also get the impression that in some cases developers are unintentionally luring the PRB into the role of a development testing and consultancy service, whereas the PRB is supposed to be simply a final quality check.

> If anyone wants to offer addons that haven't gone through the PRB, they can.

Of course they can. The advantage of using the C5 MP is the exposure to a targeted market. Why should a reputable dev with a good track record who can help conserve resources be denied access to that market?


> I think the current griping has grown from addon developers shooting in the dark.

Perhaps, but this by no means the first time PRB issues have arisen. There are apparently issues from both a dev and reviewer perspective that aren't being addressed.

Your point about some devs potentially using the PRB as a beta testing service is a good one. That's actually the impression I get (never having been part of the PRB). It's also part of the reason I can't justify devoting time to the PRB right now. Heck, I spend enough time providing feedback and bug reports to vendors of software for which I've paid!

-Steve

> Of course they can. The advantage of using the C5 MP is the exposure to a targeted market. Why should a reputable dev with a good track record who can help conserve resources be denied access to that market?

I said the same thing as @JohntheFish not because I wanted to say "If you don't like it tough", but instead to allow you to realize that a very tightly controlled marketplace is actually a huge opportunity to sell outside the marketplace(and the 25% cut).

What history has shown us about markets is that the consumer stops looking when they find what they need or want, but continues to search if that need or want is not serviced. I'm not saying you should sell outside the marketplace, but you shouldn't look at a very tightly controlled marketplace as a terrible thing. There are great opportunities in both situations. Neither are bad really. The only bad one would be if concrete5 went the Apple Store route and banned outside addons and since concrete5 is MIT licensed that's pretty much impossible.

Heck, if you think the marketplace stinks you can literally start your own and if you provide customers and developers with a better service you could stand to make a bunch of cash.

I'm not saying if you don't like it "tough" as much as I'm saying I honestly think a really tightly controlled marketplace will in the end allow devs and users more freedom in the long run as everyone will eventually get used to going outside of it sometimes.

Case and point look at Miser. I see that installed on more sites I come into than any other addon and that isn't in the marketplace.

The sourest lemons lead to the best lemonade, but you still need to add your own sugar.

mkly,

Your points are well taken. And as the popularity of C5 continues to grow, it might be worthwhile to pursue other selling venues.

As for the Miser mod, it's a great example of quality development and support. Phallanx, the developer, obviously uses it himself and keeps it up to date; but he's also very responsive to support requests. And despite the mod coming with a license that some have characterized as weird or non-standard, the product and support speak for themselves.

-Steve

If its any consolation, in my experience the PRB usually turn around a theme (cant speak for add ons) in about a week, my second theme was even quicker.

My personal feeling is the Blog here is an underused tool, not everyone watches Totally random, so a few sentences in the blog to keep people informed would be welcome I think.

I'll take a look at every theme today.

I just want to add something that I think gets lost with the presence of the marketplace. There is nothing at all stopping you from making your addons available and selling them outside the marketplace. Concrete5 doesn't prevent unapproved addons from installing and running like the Apple App Store does.

The marketplace is great, but you aren't forced in any way to go through it to get your addons installed on concrere5 websites.

I agree, that would be an option. However, I feel that I wouldn't be able to provide the same level of support to customers that I would here, and that my customers might feel the same way if they bought them elsewhere.

There's already been a lot of activity in the PRB since this post started. It looks like a few members have thrown some considerable energy at it over the weekend. We will do some passes our selves over the next two weeks and end the year with nothing in there. With the great volunteer energy from the community and some interface improvements we should be able to easily turn over submissions faster.

The changes we've been talking about are:
1) Making it so members can click a button to vote that it works. Just like our bugs system, it makes it much easier to approve stuff quickly.
2) Improving the list view to sort by activity, votes, status.
3) Adding a status.
4) Adding a health bar for the process on each submission
5) Having a fancy ajax displayed check list of items to pass - be they automatic, approved by member, or approved by us.
6) Automatically removing old submissions faster.
7) Giving PRB members a free license of any add-on/theme they are part of approving, along with more karma and badges.
8) Boot inactive PRB members, include a wait list to join. Have some automatic rules around frequency of contribution.


Any other thoughts?

It seems that failing on installation is a popular grief. Maybe a special vote button just for installation testing. So a PRB memeber who doesn't have time for a full test can do a quick install and vote it doesn't bork for some quick karma etc.

I like it all except the free licence part of (7). Despite potentially benefiting from it personally, it will be impossible to manage fairly and open to unintentional (and intentional) abuse. It could easily become a massive area of contention.

One PRB member could give a detailed test and analysis. Another could simply have a quick look and say 'looks nice'. Another could put a lot of work into seeing a new addon not install and so just give it the 'not install' vote. What if an addon gets 10 PRB members downloading and testing (yipee!, that's exactly what you are asking for), all in the dark as to what the others are doing. They come back to post their comments, only to find that someone else has in the meantime posted exactly what they were going to say.

How do you draw a line between these without introducing another level of subjective review, where you are constantly reviewing the PRB members and the worth of their effort and comments?

If you have to offer licence rewards, maybe have one free licence for each approved addon and a karma lottery purely within the PRB. I only offer this suggestion as a watered down compromise, personally I don't even think that would be a good move.

Some of my thoughts that come into my mind about the PRB improvement ideas:

Button to say works/doesn't:
What about when a fix comes in that fixes the issues that people clicking "doesn't work" have experienced? The add-on stays with a "bad" reputation and if no one will have any time to test it another time, it will end up with "30 days and no activity" sign and will be deleted from the marketplace. There might be e.g. add-on with 10 "no" votes and 0 "yes" votes and after some fix, it might work ok but for you guys (who do the final approval) it looks like it doesn't if you just look at the votes.

Helpful insights vs. "it works"
I really like that after I posted my message, there started to come some action to one of those add-ons. The post was really helpful and pointed out usability issues that we didn't even think about. Additionally, it pointed out one possible lacking feature that might be good to add there. To me (and us) this is really 100x more helpful than just some comment "it works and installs ok". How do you separate these comments from others? Maybe provide add-on developers the same kind of "helpful" button for those posts that you currently provide here at the forums "mark best answer"? However, in the PRB, you should be able to choose multiple helpful posts.

Giving free licence
I think here it is also crucial to separate people as JohntheFish stated out. Maybe the "helpful" posters would get a licence but those stating that "it works" not... However, if the add-on is perfect (well, I'd like to hear about "perfect software" in my lifetime) and has no issues, then giving insightful comments might not be that trivial.

In the helpful posts there is still the problem that JohntheFish also pointed out: what if there are e.g. 2 people testing at the same time and both have the same comments? The one wins, who gets to post their comments first.

Just some thoughts.

And about the current situation that I read FOR THE FIRST TIME in this post. Add-ons don't get approved if they don't work with C5.5? I somehow could understand why, but in an email that you (frz) answered to me recently, you said that this decision would be up to developers when they are ready to provide a version for C5.5? I think this is the correct way to go because e.g. of the 3 add-ons we have in the PRB, 2 are eCommerce extensions and currently there isn't a working version of eCommerce for C5.5 we could test/develop with.

Thanks frz for the long comments, it's really great to see that you're also trying to improve things around in the PRB.

I also feel that it's good to have such a marketplace with quality add-ons that are guaranteed to work. I just also want to emphasize what Shotster said earlier. The current approval policy really guarantees that the initial 1.0 version works and installs ok. In an update when additional features gets added to the package, e.g. version 1.1 might be totally broken because the PRB process does not require developers to go through it again. I don't feel that there should be same kind of process for each update (think of the workload and also think of minor bug fix updates) but just saying that the PRB process does not guarantee 100% that the add-on will work and will not break people's sites.