The theory of evolution (in a nutshell)
Descent with modification
Charles Darwin described evolution as ‘descent with modification’ meaning that offspring are generally like their parents, but with small differences. And his crucial idea was that those offspring with favourable differences/variations will generally survive (and reproduce) better than those without – which is natural selection. Repeated operation of this process (producing variations and selecting favourable ones) means that over the course of many generations small changes can accumulate, and overall significant changes can occur.
There is no doubt that in general terms the above is true; but we need to unpack it to have a clearer understanding of what is going on, and to identify potential limitations.
Variation within species
For all naturally occurring species, their members are not uniform but differ from one another, usually in many respects. These differences may be in obvious morphological features, perhaps overall size or of some part(s) such as length of limbs, or colour; or they may be less noticeable such as rate of maturing, ability to metabolise (or not) particular foods (especially in bacteria), blood groups, or susceptibility to diseases.
Variations arise because individuals have different genetic compositions (genotype): For any gene with a given role, within a species there can be several different forms (called alleles) of this gene, although a particular individual will have only one or two in their genotype. It is the range of alleles that provides variability (the capacity for a range of variations to exist):
Mixing and segregation of genes/alleles occurs at reproduction, especially sexual reproduction where offspring derive a random mix of alleles from each parent. Because there are a large number of genes (typically tens of thousands), many with several possible alleles, there are an astronomical number of possible allele combinations – far larger than the number of individuals in a species’ population (except for e.g. bacteria and some insects). So usually each individual within a species has a different subset from the available alleles; hence all individuals are slightly different from each other.
Competition and Natural Selection
Every individual must acquire resources to live, grow, and reproduce. However, nature’s resources are limited, so all individuals must compete for them – to obtain food and/or to avoid becoming another’s food. It follows that those individuals that are better equipped to survive will usually live longer and (more significantly from an evolutionary point of view) generally reproduce more successfully (have more offspring).
Where that survivability is due to heritable variations then the genes/alleles responsible for those variations will tend to be more prevalent in the next generation, and in general the variations themselves will tend to be perpetuated (the link between genes (genotype) and morphology (phenotype) is not always straightforward).
This is natural selection in action: reproduction leads to members of a species having a variety of gene combinations (genotypes), conferring a range of variations (phenotypes); those that enhance survival reproduce preferentially, leading to an increase in the next generation in the frequency of alleles associated with those variations.
The operation of natural selection can lead to both adaptation and speciation.
Probably the best-known example of adaptation through natural selection is industrial melanism in the peppered moth:
Before the 19th century the peppered moth (Biston betularia) had pale-coloured wings; but then it started to be found with dark-coloured wings, and throughout that century the proportion with dark wings increased until by its end they were almost all dark in at least some geographical areas. This change was attributed to the effects of industrial pollution: the moths rest on tree trunks; before industrialisation many trunks were covered with pale-coloured lichen so pale-coloured moths were better camouflaged; but lichen died off due to industrial air pollution, leaving darker-coloured trunks against which dark coloured moths were better camouflaged. As the moths had genes (alleles) for both light- and dark-coloured wings the effect of the change in environment was to switch the preferred gene from one to the other. Interestingly, since the 1950s, as air quality has improved and lichen has returned, there has been a return of the pale form as the moth has adapted again to the changing environment.
Adaptation by the moths was a relatively minor change involving just one or a few genes and able to take place over a short period of time.
Continued operation of this process – production of variations, and selection of favourable ones – can enable a species to adapt progressively to its environment, sometimes involving significant morphological change. Well-known examples of this are the various types of Galapagos finches with their variety of beak size and shape, each adapted to its particular mode of feeding.
The Galapagos finches also illustrate speciation.
Especially where a species has an extensive geographical range, it may be that the environment (or other factors) is significantly different in different parts of its range, such that different variations are preferable in different parts of the range. Where this occurs, and if individuals tend to reproduce with others in their own locality rather than from further afield, then the species can become significantly different in different parts of its range – i.e. distinct races can emerge. If this process persists then a species can split into two or more distinct species.
(Although traditionally the criterion for a species is that its members can interbreed, in practice species are generally considered distinct if there are significant morphological differences between their respective members, especially if they are also in different geographical locations.)
The classic example of speciation is the Galapagos finches: they are usually classified into 14 species, grouped into 3 genera; and all are thought to have derived from a common ancestor which emigrated from the South American mainland (see Figure 2). The evolution of the finches illustrates speciation, and probably anagenisis as well.
What's happening at the genetic level?
So far as the moths are concerned it is generally recognised that both the pale and dark forms occurred from long before industrialisation, and all that happened was a shift in which form was prevalent. That is, it is clear that no new genes arose.
For the finches it is also likely that all of the adaptation and speciation occurred solely through segregation and selection of genes that were already present in the common ancestor, i.e. that this also did not involve the origin of new genes. (In a similar way to how the many dog breeds have been derived from the wolf by artificial selection, see below.) For example, a recent genetic study has correlated variants of a specific gene with size of the finch beak, presuming they were selected from a common ancestor, and raises the question of when the different variants arose.  Further studies will shed more light on this.
The effect of selection on genetic variability
As mentioned above, the effect of selection is to increase the frequency of those alleles associated with favourable variations. The other side of this coin is that alleles associated with less favourable variations will become less frequent – even to the point of being lost altogether from the species’ population. When this happens there is a decrease in the number of alleles available, and a consequent reduction in variability. (This decreases the number of possible variations and reduces capacity for future adaptations.)
All of the above – the occurrence of natural selection, and consequent evolution in terms of adaptation and speciation by selection and segregation of alleles/genes – I agree with and accept as substantially true.
New genes and variability
A crucial assumption of the theory of evolution is that, whilst alleles (or even genes) are lost through selection and segregation, new ones arise (mainly through mutation). Hence there is always genetic variability available for new morphological variations to arise and enable evolution to continue indefinitely. Not just continue in a minor sense – such as a change in colour, size or shape of a feature – but enable significant progress, involving substantial novel structures such as eyes, wings or flowers – ultimately enabling the evolution of higher (more complex) organisms from lower (simpler) forms. Indeed, this has to be assumed otherwise evolution would grind to a halt, in fact could never have got started. However, the evidence is clearly against this happening (e.g. see the biochemical hurdles to new genes), and this is one of the key arguments against the evolution of ‘higher’ from ‘simpler’ forms of life.
Domestic breeding / Artificial selection
To help introduce and explain the concept of Natural Selection, Darwin compared it with domestic breeding.
A breeder wanting to produce a variety (whether of animal or plant) having a particular characteristic selects those individuals having that characteristic to at least some degree, and uses those to breed the next generation, hoping that the characteristic will at least be retained, and preferably enhanced (if both parents have the desired trait) in their offspring. The success of this approach is evidenced by the many varieties of both animals and plants that have been bred for particular purposes, such as cereal crops from wild grasses. And the wide variety of dog breeds[d], all derived from the wolf, illustrate the number and range of characteristics that can be selected; and indicate the extent of genetic variability present in most natural species.
However, breeders are well aware that, although significant development can usually be achieved in the first few generations, the degree of change rapidly falls off and approaches a limit. It is for this reason that many of Darwin’s contemporaries (some of whom were also considering the so-called ‘species question’) did not think that substantial evolutionary-type progress could be made through a process analogous to domestic breeding, because they were all too aware of its limitations.
With our modern knowledge of genetics we understand that what is happening in (conventional) domestic breeding is the selection of subsets of alleles/genes from those available in the initial species (or other variety(ies) used as a starting point for a breeding programme). Once all the alleles/genes providing the desired trait are selected in and those countering it are excluded, the trait cannot be developed any further.
Richard Dawkins comments on this in The Blind Watchmaker:
If anything, selective breeders experience difficulty after a number of generations of successful selective breeding. This is because after some generations of selective breeding the available genetic variation runs out, and we have to wait for new mutations. 
And that is the important point: selective breeding is constrained by the genetic material available in the starting species. And, I suggest, this is also true of any evolution that is known to occur. Just as domestic breeding must ‘wait for new [useful] mutations’ so must naturally-occurring evolution.
The crucial question is: Do useful new genes actually arise?
This is discussed in the biochemical reasons against genes evolving.
Notes display in the main text when the cursor is on the Note number.
1. Nala Rogers, Evolution of Darwin’s finches tracked at genetic level, Nature News (21 April 2016), doi:10.1038/nature.2016.19795
2. Richard Dawkins (1991, first published 1986), The Blind Watchmaker with Appendix, Penguin Books, chapter 9, p247; italics in the original.
Graphics are by David Swift unless otherwise stated.
Background image for banner from https://pixabay.com/en/jungle-bird-branch-wildlife-fauna-1209325/ (no attribution required).
a. Repeating patterns of mimicry. Meyer A, PLoS Biology, Vol. 4/10/2006, e341 doi:10.1371/journal.pbio.0040341, open-access article distributed under the terms of the Creative Commons Attribution License.
b. Image ID 82484187 © Dr Ajay Kumar Singh | Dreamstime.com
c. Image (https://upload.wikimedia.org/wikipedia/commons/b/b7/Lichte_en_zwarte_versie_berkenspanner.jpg )by Martinowksy from nl [GFDL (http://www.gnu.org/copyleft/fdl.html) or CC-BY-SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0/)], via Wikimedia Commons.
d. Image ID 51407706 © Junojess | Dreamstime.com
Page created March 2017.