Back with a bang

Well, after almost a year of university-/work induced slacking, I'm finally going to work some more on Gentoo PHP.

Speaking of which, I've just closed Bug #324665 (Merge dev-php5 and dev-php) as RESOLVED FIXED. After a year of procrastination, I've finally whipped up a script to migrate the complete dev-php5 category into dev-php. Thanks goes to olemarkus and darkside for checking my script and suggesting ways to go about this.

If there are any issues with the migration, please comment here or on the bug.

P.S.: Yes, I know the overlay is broken because of this. Neither Ole nor me have any use for it at the moment. Plus, merging user-contributed stuff (thanks jamie!) from the overlay to the gentoo-x86 cvs tree is just plain painful. So unless we can keep Gentoo PHP development completely in the overlay and make the push as simple as git pull overlay, you probably won't see much action in the overlay.

php-5.3.3 sqlite behaviour

In this post I like to point out to you Bug #340059. It's about using sqlite3 without getting sqlite2 forced on you by portage.

The problem here was that to enable pdo-sqlite, the sqlite driver for PHP Data Objects, you had to enable USE="sqlite". This was fine when there was only one sqlite USE flag. But with both sqlite and sqlite3 selectable, it became an odd choice. PHP Data Objects only fully support sqlite3 in the first place!

So I changed that today and made PDO support for sqlite dependent on USE="sqlite3". The update should now be on a mirror near you. By the virtue of eblits, it is available for all php-5.3 versions we currently have. Simply remerge php (USE="-sqlite sqlite3" emerge -v php) to see the effect - an emerge --depclean should now unmerge sqlite-2* if PHP was the last application requiring it.

And while Dessa kindly pointed out to me Bug #249418, which states that USE="sqlite" should enable either sqlite2 or sqlite3 support and prefer sqlite3 (thus obsoleting the sqlite3 USE flag), I'm still in favor of the current solution.

For one, PHP upstream still enables both sqlite2 and sqlite3 by default. And if you install a Gentoo php without any changes, you'll get sqlite2 and sqlite3, too, because we follow upstreams defaults.

But heads up: future versions of PHP-5.3 will throw E_DEPRECATED notices if you continue to use sqlite2 via sqlite_open() as per this post. Support for it will be gone in future minor versions (whether they will be named 5.4 or 6...)

Fixing /usr/portage/profiles/default/linux/make.defaults

This is a post about probably (to an outsider) boring Gentoo internal development affairs. Feel free to skip or skim it, if you're only here for the PHP status updates ;)

Part of the update to dev-lang/php-5.2.14 was the change to IUSE defaults. This allowed us to provide a PHP build that matched what you could get by simply downloading and compiling the PHP upstream tarball by hand.

It also allowed fixing a long-standing papercut with the make.defaults file. In 2006, the release team discovered php was breaking the release process because certain packages required dev-lang/php to be built with certain USE-flags enabled. For lack of a better solution, the required flags were added to the top-most make.conf template available at that time.

Today, /usr/portage/profiles/default/linux/make.defaults still contains USE="${USE} cli pcre reflection session spl". This results in every linux system having these USE flags enabled by default. At least for spl this is a waste: no other ebuild besides php uses this flag.

Initially, I was going to scrap the whole line, but it turned out to be rather difficult. Not counting spl, all other USE flags are utilized by other packages beside php. So removing the make.defaults line would change their default behavior, too.

Therefore I removed "spl" from the list in make.defaults. For the rest, more consultation is needed. So if you read this and maintain a package using one of "cli pcre reflection session", please comment on this post or better yet, on bug #310383. Expect a mail on gentoo-dev about this, soon, too.

php-5.2.14 USE flag clarification

This is in response to a rescue operation I did yesterday evening, after being notified of Bug #332257. In essence, I've failed to include two USE flags (force-cgi-redirect and discard-path) which should have been there and castrated a third one (cgi).

I snuck those missing two back in and fixed USE="cgi" to also enable fastcgi, like 5.2.13 did. So if your FastCGI setup stopped working after an upgrade to 5.2.14, please resync and remerge php-5.2.14, the regression has been fixed.

A friendly user also asked me about some other changes, I only then realized I failed to document: the sybase, java-external and fastbuild USE flags are gone from 5.2.14. You'll notice that they also don't appear on php-5.3.3. This has a reason, which is probably only appearent if you follow php-bugs@gentoo.org on our Bugzilla, so here goes:

sybase

... was dropped when Bug #281316 came up. I learned that sybase-ct as provided by dev-db/freetds is a near complete stand-in for sybase. If the "near complete" doesn't work for you and you need sybase back, please comment below. But note PHP-5.3 has dropped support for that extension, so you'll need to find another way, anyway.

java-external

... has never done much to the PHP build - it only pulled in dev-php5/php-java-bridge and enabled USE="session". Now, the java-bridge package has not been available in our tree since 2007 and the official status of the "java" extension according to the PHP manual is unmaintained and dead. So I decided to drop this. If you need Java support, please head over to Zend, but I've no experience with their Java stuff, so I can't comment further.

fastbuild

... was an experimental patch to the PHP build system I dropped in a bid to provide a more "vanilla" PHP. To be honest, it wasn't in hoffies eblits for PHP-5.3, as the patch would have probably needed modifications to it in several places. And I was not in favor of adding another eblit to accomodate PHP-5.2 here. To be even more honest, I'm scared by PHP's build system so much I want to destroy and rebuild it from the ashes

News update: php-5.2.14 on the fast lane to stable

Our security team got around to filing Bug #332039, which covers all the security fixes in PHP-5.2.14 and PHP-5.3.3.

It's Gentoo policy to have a normal testing period of approximately 30 days. The period can be shortened or omitted if there are security problems in a current stable version. This is what's happening in the transition from php-5.2.13 to 5.2.14 right now. I intentionally migrated 5.2.14 from the overlay a few days after 5.3.3, so we could test-drive the new eblits backed system with 5.2.14 that makes maintaining PHP so much easier. While all eblits originally were designed for use in 5.3.2, I only needed to update one eblit to accommodate 5.2.14.

But there was another change with 5.2.14 that will enable me to fix bug #310383 once php-5.2.14 is stable on all architectures: 5.2.14 finally makes use of EAPI2. It heavily uses USE dependencies, and also, by popular request, includes USE defaults. This feature auto-selects USE flags if you don't disable them via /etc/portage/package.use. By default, you're now getting the same PHP features you'd get by downloading the source and just running ./configure.

The change to USE defaults also obsoletes settings certain USE flags in the default make.conf, which is what bug #310383 is about.

There's also a downside to the heavy use of EAPI2 features: we now force certain USE flags on, if you enable a specific USE flag. This is a replacement for the manual die message you've probably seen once in a while. An example of this is USE="truetype", which needs the gd (or gd-external) USE flag on. This currently results in two kinds of error messages, a nice one and a very confusing one. Observe (sorry about the wrapping here):

terra mabi # USE="readline libedit" emerge -1 php
These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild N ] dev-libs/libedit-20090923.3.0 456 kB
[ebuild R ] dev-lang/php-5.3.3-r1 USE="libedit* readline" 0 kB

Total: 2 packages (1 new, 1 reinstall), Size of downloads: 456 kB

!!! Multiple package instances within a single package slot have been pulled
!!! into the dependency graph, resulting in a slot conflict:

dev-lang/php:5

('installed', '/', 'dev-lang/php-5.3.3-r1', 'nomerge') pulled in by
=dev-lang/php-5.3.3-r1[-libedit] required by ('ebuild', '/', 'dev-lang/php-5.3.3-r1', 'merge')
(and 1 more)

('ebuild', '/', 'dev-lang/php-5.3.3-r1', 'merge') pulled in by
php

Explanation:

New USE for 'dev-lang/php:5' are incorrectly set. In order to solve
this, adjust USE to satisfy '=dev-lang/php-5.3.3-r1[-libedit]'.

While I find the fact that portage is complaining about a "slot" problem misleading, it gives a nice and clear explanation about what to do to recover. Compare to this:

terra mabi # USE="sharedext sharedmem threads" emerge =dev-lang/php-5.2.14

These are the packages that would be merged, in order:

Calculating dependencies... done!

emerge: there are no ebuilds built with USE flags to satisfy "=dev-lang/php-5.2.14[-threads]".
!!! One of the following packages is required to complete your request:
- dev-lang/php-5.2.14 (Change USE: -threads)
(dependency required by "dev-lang/php-5.2.14" [ebuild])
(dependency required by "=dev-lang/php-5.2.14" [argument])

Duh! I've no clue about the root issue by looking at the error message. Turns out USE="sharedmem" clashes with USE="threads", but you can only discover this by looking at the ebuild. I'm currently working out how to get you a more informative message.

That's the news for today. Stay tuned while we work out the details about slotting PHP to minor versions (certainly the topic of one of the next posts), which we hope to have out before PHP gets a 5.4 version.

EOL for USE=concurrentmodphp

As previously mentioned, concurrentmodphp is currently available as a USE flag for dev-lang/php. It enables you to run two versions of PHP in parallel, loaded into your apache's mod_php.

With the advent of FastCGI/fpm in PHP-5.3.3 (very soon on a mirror near you), the preferred way to run multiple versions of PHP in parallel is CGI. Not only can you run several PHP versions independently, you also get proper script isolation (running each script as a different user, if you wish so) and thus enhanced security.

This is why I've decided to end support for USE=concurrentmodphp after php-5.3.3 and php-5.2.14. Versions after those will simply stop shipping the patches required to support this USE flag. Please use the time with PHP-5.3.3 to switch to fpm, if you need the functionality currently provided by concurrentmodphp.

Using multiple PHP versions

This is another "state of php in gentoo" article. Specifically, I want to talk about using SLOTs with PHP.

The current situation was created during the switch from PHP-4 to PHP-5, one of the biggest and more drastic in terms of features. PHP-5 added a full blown object oriented system, ending PHP's dubious fame as a highly advanced template engine and easing work on large code base projects. Clearly, porting your code from using PHP-4 to 5 was a tremendous effort if you wanted to take advantage of all the new and powerful features PHP-5 offered. So the then gentoo php team decided to make it possible to install PHP-4 and 5 alongside, easing migration testing and extension usage.
What they did was to slot PHP to it's major version. This is why today you're left with dev-lang/php in one slot: "5".

In a meeting on 11th April, the gentoo php team decided to halt decisions on slotting less (i.e. removing slotting) or more (i.e. on minor versions), due to manpower restrictions. This was chiefly due to our desire to get php-5.3.2 into the tree as fast as possible and we needed every minute we could contribute to gentoo at that time to this task.

Now that php-5.3.2 is in the tree and I'm slowly killing the remaining bugs left (helped, and mostly driven, by many friendly contributors, thank you!), we can focus on other areas to improve the overall quality of gentoo's PHP offers.

So slotting became an issue once again.

In an informal meeting, we agreed that removing slotting was the easiest solution, but not the best. While slotting to minor versions increases ebuild complexity and comes with it's own problems (you need a way to select the "active" PHP installation to run for the apache module, for example), the advantages in migration testing and also for shared hosting providers, who want to offer more than one PHP version, trumped the relatively low cost of slotting.

Here's a short pro-contra overview of minor version slotting I came up with:
pro

• test your code against the most current PHP version on the same machine


• provide your customers the PHP they want

contra

• need to remove slots you don't need (or they'll take up hard disk space)


• increased complexity in configuration

That's pretty short and I'm sure you can come up with more benefits and disadvantages. Do let me know in the comments!

I'm still somewhat disappointed with the lack of "end-user" benefits in this move. If you just want to run that cool webservice your friend has been telling you about, you'll emerge php and will receive slot updates instead of the normal ones thereafter. You regularly want to run emerge --depclean, so you don't keep around PHP slots you no longer need (thanks to darkside for helping me with this one).

Another hurdle will be that we require you to "enable" the PHP version you want via eselect to benefit from the slotting. This affects for what PHP version your PHP extensions like apc or suhosin will get compiled (each minor version has a different ABI, so for each SLOT you need a special build). We're trying to make it so that not "enabling" anything will guess the correct PHP version (probably your highest slot), but that's not sure right now. You will also use eselect to switch from one php apache module to another.

Speaking of apache modules, I've another proposal: currently, Gentoo supports a highly experimental feature called concurrentmodphp. It's a USE flag for dev-lang/php and enables you to run multiple versions of php in the same apache. Usually, this results in a symbol name clash of the php modules you try to load, but with this USE flag enabled, it should work. That said, I never used it, did not test it and upstream will tell you to rebuild without it if you do and run into problems. It's in no way supported neither by Apache or PHP. What is supported is running multiple versions of whatever you like via cgi (fpm, for that matter). So if you want that functionality, you can get it anyway, without causing maintenance problems for the team.

So here's the proposal: I will drop the concurrentmodphp functionality starting in August, if nobody objects with a technical argument on why to keep it. I'd like to hear cases where concurrentmodphp solves a problem fpm can't handle.

Anyway, if you want to help with the slotting efforts, please visit #gentoo-php on freenode, tell us to pull your ideas from your git repository (you can clone your local copy off of our slotting branch) or open bugs explaining why such and such change in said branch won't work, kills kittens or is otherwise undesirable. As always: comments welcome!