This is a development env setup based on docker suitable for php.
The docker setup uses the docker-sync with unison strategy this allows to avoid the performance penalty of default docker use on OSX.
For linux users, it is smooth sailing.
WARNING: THIS IS NOT TO BE USED IN PRODUCTION
You need to have docker for mac installed.
You will need to install a gem for docker-sync, fswatch, unison and macfsevents,
by executing the following commands from your terminal
Docker-sync 0.2.0 now supports the use of environment variables for configuration, you can then install it just by running
gem install docker-syncAnd then you continue with installing the rest of the prerequisites
brew install unison
brew tap eugenmayer/dockersync
brew install eugenmayer/dockersync/unoxDocker runs the daemon as root, and all the files created by volumes by default will belong to root:root, which is,
well, bad... And I'm going to ignore the security implications, and just go with usual dev business, that you would
have to run your IDE/editor with elevated privileges.
Luckily the answer exists, and it is user namespaces which are present since Linux 3.5, and stable since Linux 4.3. Depending on your distribution, the steps needed to enable user namespaces may vary, and as such are not covered by this document, google is your friend.
First you need to find out your user id and group id, this document will assume they are 1000:1000.
Edit /etc/subuid to look along the lines of
vranac:1000:1
vranac:100000:65536Replace vranac with your username, and replace 1000 with your userid in the first line.
The second line, if it exist, should match your username, the numbers themselves are of less importance.
Then you should edit /etc/subgid to look along the lines of
vranac:1000:1
vranac:100000:65536Replace vranac with your username, and replace 1000 with your userid in the first line.
The second line, if it exist, should match your username, the numbers themselves are of less importance.
The changes you made read as “let user vranac use his own uid as well as 65535 uids, starting at 100000 and making the total of uids to 65536”.
And this won’t break anything.
When starting docker with docker daemon with user namespace remap to vranac, docker will parse the subuid and subgid files for
vranac, sort all read entries by growing start id and generate kernel userns mapping rules.
Without diving too much into the details, this will generate the following rules in /proc/[PID]/uid_map:
0 1000 1
1 100000 65535Which should look familiar. This structures looks like the one above, but the meaning it slightly different. This time, it reads as:
“Let uid 1000 outside the container act as root inside the container” “Let the 65535 uids starting with 100000 outside the container act the 65535 uids starting with 1 inside“ In other words, 1000 will be 1 and 100002 will be 3.
So to enable this, the final step is to edit the /etc/docker/daemon.json to look along the lines of
{
"userns-remap": "vranac"
}You need to replace vranac with yout username.
After this, restart the docker daemon, and from now on, all your host mounted volumes will be under your user:group
When the docker moved to the new engine, one of the things to go is docker0 network interface. To quote the docs on networking
I want to connect from a container to a service on the host The Mac has a changing IP address (or none if you have no network access). Our current recommendation is to attach an unused IP to the lo0 interface on the Mac; for example: sudo ifconfig lo0 alias 10.200.10.1/24, and make sure that your service is listening on this address or 0.0.0.0 (ie not 127.0.0.1). Then containers can connect to this address.
What this essentially mean is that you don't have a network address to target with xdebug.
But there is a workaround... You can either execute the following in your terminal to create a loopback interface alias to an ip address
sudo ifconfig lo0 alias 10.254.254.254 255.255.255.0You would, of course, have to reexecute it on restarts, as this is not a permanent solution.
Or if you want a more permanent solution, you can copy the com.docker_alias.plist file from osx-config to
/Library/LaunchDaemons/com.docker_alias.plist and from then on, on every reboot the loopback
will be setup.
To verify that everything has been setup properly, execute ifconfig from terminal and the output
should look along the lines of this
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
options=3<RXCSUM,TXCSUM>
inet6 ::1 prefixlen 128
inet 127.0.0.1 netmask 0xff000000
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
inet 10.254.254.254 netmask 0xff000000 <--- here you can see it has been setup properly
nd6 options=1<PERFORMNUD>If you want to use a different ip, go ahead, but make a note of it, and put it into your .env for HOST_IP key.
The plist file is based on this gist
For linux users running ifconfig will give you the list of defined network interfaces, you want the docker0 info.
The output would look something like this
docker0 Link encap:Ethernet HWaddr 02:42:43:64:dc:98
inet addr:172.17.0.1 Bcast:0.0.0.0 Mask:255.255.0.0 <-- This is what you want
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)The ip address is 172.17.0.1.
So now you take the ip address you got, and put it into your .env for HOST_IP key.
PhpStorm integrates docker api by using a port, but, from recenlty docker only allows api interaction via sock.
PHPStorm now allows direct referencing of sockets. To configure PhpStorm you will need to do the following:
- open preferences
- under
Build, Execution, DeploymentselectDocker. - if there is no Docker instances defined, create one by clicking on the
+icon, and give it a name - In Api URL enter
unix:///var/run/docker.sock - clear the certificates folder input
- set the docker compose executable to
/usr/local/bin/docker-composeor wherever it is (you can find out by runningwhich docker-compose) - make sure
Import credentials from docker machineis checked off - Click OK
To solve this, you will need to install socat. Installation instructions for it on linux differ by your particular flavor. To install socat on OSX, you need to execute the following
brew install socatSo once you have socat installed, you would need to start it, by running
socat TCP-LISTEN:2375,reuseaddr,fork,bind=localhost UNIX-CONNECT:/var/run/docker.sockIf you want this to be setup automatically on each start, for linux create an init script and place the above command in it.
On OSX you need to do the following steps:
- copy the
docker.sock.plist.distinosx-configdirectory todocker.sock.plist - in
docker.sock.plistupdate the UserName and GroupName - copy
docker.sock.plistto~/Library/LaunchAgents/ - start with launchctl load ~/Library/LaunchAgents/docker.sock.plist
And now on every restart the socat will be automatically setup.
To configure PhpStorm you will need to do the following:
- open preferences
- under
Build, Execution, DeploymentselectDocker. - if there is no Docker instances defined, create one by clicking on the
+icon, and give it a name - In Api URL enter
tcp://localhost:2375 - clear the certificates folder input
- set the docker compose executable to
/usr/local/bin/docker-composeor wherever it is (you can find out by runningwhich docker-compose) - make sure
Import credentials from docker machineis checked off - Click OK
PhpStorm will now be able to interact with docker api, and you will be able to setup remote interpreter etc.
If you are using this as a dropin environment in your project via git submodule or subtree.
To use the new docker development setup you should add it as a git subtree. The process is simple, you need to execute a few commands to get started. Adding the subtree as a remote allows us to refer to it in shorter form:
git remote add -f docker-dev-setup https://github.com/code4hire/docker-dev-setup.gitNow we can add the subtree, but now we can refer to the remote in short form:
git subtree add --prefix docker docker-dev-setup master --squashThe command to update the sub-project at a later date becomes:
git fetch docker-dev-setup master
git subtree pull --prefix docker docker-dev-setup master --squashYou need to symlink the docker/bin to bin in your project root.
You can do this by executing
ln -sf docker/bin binCopy the docker/.env.dist to .env in your project root.
PAY ATTENTION TO THE COMMENTS and edit the .env to modify the following:
APP_PATHDATA_PATHSYNC_NAMEDOCKER_SYNC_LOCATIONCOMPOSE_PROJECT_NAMECOMPOSE_FILE
Pay special attention to DATA_PATH, DOCKER_SYNC_LOCATION and COMPOSE_FILE depending
if you are using docker-sync or not, and depending on your setup, the paths need to be adjusted.
If you are not using docker-sync, make sure that COMPOSE_FILE has no reference to docker-compose-dev.yml
If you are using docker-sync, you can start it in the terminal by executing
bin/docker-sync-daemon.sh startAfter this is done, build your containers for the first time by executing
bin/docker-compose.sh buildCreate application directory, if not already present, by executing
mkdir applicationOnce your .env has been edited right, and your containers have been build for the first time, and docker-sync is running
(all these steps were described above), you can start this project by running
bin/docker-compose.sh up -dthis will start up all your containers and services in detached mode.
You can check on their status by running
bin/docker-compose.sh psThe following services have been setup for use:
Optional:
Official docker images are used whenever possible, but some had to be built from scratch, or a popular one was used
For php and php-fpm, the official images available at docker hub are being used.
For node, the official images available at docker hub are being used.
For mailhog, the official images available at docker hub are being used.
For nginx, alpine 3.5 official image available at docker hub is being used, and then nginx is installed on top of it.
For elk, the images available at docker hub are being used.
For ngrok image available at docker hub is being uses.
For Selenium standalone, the images available for chrome and firefox are being used
- copy .env.dist to .env
- edit the .env variable values and follow instructions in the comments
- edit the docker-sync.yml and give syncs unique names OSX ONLY
- edit the
docker-compose-dev.ymland update the sync names with the ones you set in the docker-sync.yml OSX ONLY
If you are starting the setup for the first time, or you changed some info in the dockerfiles, or configs, you need to (re)build the containers and images by executing
docker-compose buildThis command will download all the images, build them up into containers and prepare everything for you. Depending on your machine specs and internet connection, this can take anywhere beteen 10 and 30 minutes (maybe more), as usual YMMV.
If you are on OSX all you need to do now is to execute
docker-sync-stack startIf this is your first run, all the images will be build, linked and started, depending on your codebase size unison might take a while to sync them all up, as explained here
This basically executes two commands as one, it starts the unison sync container with your mounts, and starts the defined containers, this is equivalent to these two commands
docker-sync start
docker-compose -f docker-compose.yml -f docker-compose-dev.yml upAs this covers the linux users as well, I will contine with the second form going forward.
So, for OSX users in one terminal (tab) run
docker-sync startand in another terminal (tab) you should run
OSX: docker-compose -f docker-compose.yml -f docker-compose-dev.yml up
linux: docker-compose upOSX needs two config files to setup the volumes correctly, linux does not have the performance penalty,
so it can run with the default docker-compose.yml.
What this does is run all the containers but in foreground, if you want to run them in background
(aka detached mode), you need to add the -d switch
OSX: docker-compose -f docker-compose.yml -f docker-compose-dev.yml up -d
linux: docker-compose up -dSo now you need to stop the containers and services.
It is quite easy, if you are running docker-sync-stack or are NOT in detached mode, all you need to do is
press ctrl+c this will stop the containers, and might throw some errors (that is ok, and ymmv).
If on the other hand, you are running in detached mode then you need to execute the following
OSX: docker-compose -f docker-compose.yml -f docker-compose-dev.yml stop
linux: docker-compose stopTo cleanup, on OSX, if you run the whole stack at once, you need to execute
docker-sync-stack cleanon the other hand if you are running docker-sync separately from the containers, you need to run
docker-sync cleanand then for linux and osx, you need to run
OSX: docker-compose -f docker-compose.yml -f docker-compose-dev.yml down
linux: docker-compose downAt the end of all this, you will have your syncs cleaned up, and your containers stopped and destroyed.
If you need to specify the php version that will be used with cli (default is 7.1), you can set
the PHP_VERSION in the .env file
In the bin directory there are some shell script helpers to make your life easier.
The log files from nginx, php-fpm, and xdebug are being dumped into data/logs directory.
The mysql database files are located in data/mysql directory, so your db data can survive accidents.
The redis data files are located in data/redis directory.
As you have php 5.6, 7.0 and 7.1 at your disposal, the port setup for php without debug is 80 + php version, meaning 8056, 8070, 8071, and for debugging you have the 90 + php version ports, meaning 9056, 9070, 9071.
Your application will point to url localhost + port you need, for example to use php 7.1 without debug,
you would point your browser to localhost:8071
If you need to have your website debugged with php 7.1 xdebug, use url localhost:9071
By default the idekey for xdebug is docker-xdebug, you can change this in the .env file under IDEKEY,
and then ofc restart the containers.
In PhpStorm, go to preferences, then Languages & Frameworks -> PHP -> Servers, and add a new server for php 7.1
named docker 7.1, for the host set localhost, for port set 9071, check Use path mappings, and lastly
map application in your directory to /application as absolute path on server.
Add another server for php 7.0 named docker 7.0, for the host set localhost, for port set 9070,
check Use path mappings, and lastly map application in your directory to /application as absolute path on server.
Add another server for php 5.6 named docker 5.6, for the host set localhost, for port set 9056,
check Use path mappings, and lastly map application in your directory to /application as absolute path on server.
In the Edit Configurations add a new PHP Remote Debug, select docker + php version as server, and set docker-xdebug as
ide key, or whatever else you set in the .env file.
You should repeat the process to cover the other versions of php available if needed.
To send emails to mailhog, use mailhog as smtp/mailer host, and use port 1025
When you need to access the mailhog web ui, you need to open url localhost:8025 in your browser
To access the RMQ management plugin you need to open url localhost:15672 in your browser and
use admin/admin as username/password.
This can be modified by editing the docker-compose.yml
The optional containers are in docker-compose-project.yml
If you want to use any of them, copy the docker/docker-compose-project.yml to your project root.
Uncomment the containers you need, and edit your .env COMPOSE_FILE values to look like
COMPOSE_FILE=./docker-compose.yml:./docker-compose-project.yml:./docker-compose-dev.ymlAfter that rebuild your images
bin/docker-compose.sh buildand start as usual.
To access the ELK stack, you need to open url localhost:8100 in your browser.
By default this container has been commented out from the docker-compose-project.yml.
When you want to expose a local server behind a NAT or firewall to the internet.
By default this container has been commented out from the docker-compose-project.yml so it is not
started by accident.
When you need it, you will need to uncomment the code, edit the docker-compose-project.yml lines 13-14
# environment:
# HTTP_PORT: nginx:9071to match the port on nginx you want to expose. Then restart the containers
To use selenium containers, uncomment them in docker-compose-project.yml and lines 47-54 so php cli
will start them up when needed.
The url of the selenium hub/host is http://selenium-chrome:4444/wd/hub or http://selenium-firefox:4444/wd/hub
as we are using these inside of docker network, the container names get resolved, and you use ports 4444.
If you want to use the selenium containers from outside of docker, the hub/host url is
http://localhost:49338/wd/hub or http://localhost:49339/wd/hub.
As we are using the debug versions of the container, each of them has a vnc installed.
To access them use ports 5901 and 5902 for chrome and firefox respectively.
The password is secret.
Running sylius:install might cause the OOM exception, if this happens, run every part of the install as separate command