2.4. Subgroups and Cosets

This page explains the concepts of subgroups and cosets, using concrete examples of point groups and space groups to avoid overly abstract discussions.

Subgroups

On the “2.2. Concept of Groups, Point Groups, and Space Groups” page, we created new groups by adding symmetry operations one after another. The newly created groups still contain all the elements of the original group. In general, when a subset of a group satisfies the group axioms, it is called a subgroup. Conversely, when the whole set that contains a group as a subset also satisfies the group axioms, it is called a supergroup. For example, point group $2$ is a subgroup of point group $2/m$, and point group $2/m$ is a supergroup of point group $2$. In short, groups have hierarchical relationships.

The concept of subgroups is useful for understanding phase transition phenomena in crystals. This is because when a crystal undergoes a phase transition due to some change in external conditions (e.g., a decrease in temperature), the space groups before and after the transition may have a supergroup/subgroup relationship. Such phase transitions are commonly observed in cases where atomic positions change only slightly (displacive phase transitions), and as a result, twin structures and antiphase structures are formed. The concept of subgroups provides clues for analyzing such structures.

Among subgroups, there are those called conjugate subgroups and normal subgroups. Let us explain them briefly.

Conjugate Subgroups and Normal Subgroups

Suppose we have a group $G$ and its subgroup $H$. Choose one element from group $G$ (call it $g$), and for all elements of subgroup $H$ (say $h_n$), compute the set formed by $g^{-1}h_ng$, which we denote as $H’ = g^{-1}H g$. It turns out that $H’$ is also a subgroup of group $G$. The relationship between $H’$ and $H$ is called conjugacy¹, and we say that $H’$ and $H$ are conjugate subgroups. Furthermore, if $g^{-1}H g=H$ holds for all elements ($g$) of group $G$ (i.e., if $g H = H g$ holds), then $H$ is called a normal subgroup. This may seem confusing, so let us look at concrete examples below.

For example, point group $4$ has the following elements $h_0, h_1, g_0, g_1$, and selecting only $h_0, h_1$ gives point group $2$. Here, $e$ denotes the identity matrix (identity transformation).
$$h_0=e=\begin{pmatrix}1&0&0 \\ 0&1&0 \\ 0&0&1 \end{pmatrix} ,\ \
h_1=\begin{pmatrix}-1&0&0 \\ 0&-1&0 \\ 0&0&1 \end{pmatrix} ,\ \
g_0=\begin{pmatrix}0&1&0 \\ -1&0&0 \\ 0&0&1 \end{pmatrix} ,\ \
g_1=\begin{pmatrix}0&-1&0 \\ 1&0&0 \\ 0&0&1 \end{pmatrix}
$$In this case, the following relations hold: $g_0^{-1} h_0 g_0 =h_0, \quad g_1^{-1} h_0 g_1= h_0, \quad g_0^{-1} h_1 g_0 = h_1, \quad g_1^{-1} h_1 g_1 =h_1$. Therefore, we can say that point group $2$ is a normal subgroup of point group $4$.

Point group $3 2$ has a three-fold rotation axis and three two-fold rotation axes perpendicular to it. It consists of the following six matrices as elements:$$
\begin{array}{lll}
t_0=e=\begin{pmatrix}1&0&0 \\ 0&1&0 \\ 0&0&1 \end{pmatrix} ,&
t_1=\begin{pmatrix}-\frac{1}{2}&-\frac{\sqrt{3}}{2}&0 \\ \frac{\sqrt{3}}{2} & -\frac{1}{2}&0 \\ 0&0&1 \end{pmatrix} ,&
t_2=\begin{pmatrix}-\frac{1}{2}&\frac{\sqrt{3}}{2}&0 \\ -\frac{\sqrt{3}}{2} & -\frac{1}{2}&0 \\ 0&0&1 \end{pmatrix},\\
a_0=\begin{pmatrix}1&0&0 \\ 0&-1&0 \\ 0&0&-1 \end{pmatrix} ,&
a_1=\begin{pmatrix}-\frac{1}{2}&\frac{\sqrt{3}}{2}&0 \\ \frac{\sqrt{3}}{2} & \frac{1}{2}&0 \\ 0&0&-1 \end{pmatrix} ,&
a_2=\begin{pmatrix}-\frac{1}{2}&-\frac{\sqrt{3}}{2}&0 \\ -\frac{\sqrt{3}}{2} & \frac{1}{2}&0 \\ 0&0&-1 \end{pmatrix}
\end{array}$$ The set of $e$ and $a_0$ forms a subgroup. Similarly, the set of $e$ and $a_1$, and the set of $e$ and $a_2$ also form subgroups. Let us call them subgroups $A_0, A_1, A_2$, respectively. Although the directions of their principal axes differ, all of them correspond to point group $2$ in symbol notation. However, $A_0, A_1, A_2$ are not normal subgroups. For instance, for $A_0$ (the set of $e$ and $a_0$), we have $ t_1^{-1} \ a_0 \ t_1 = a_1 $. That said, $A_0, A_1, A_2$ are not entirely unrelated to each other. As already shown, since $ t_1^{-1} \ a_0 \ t_1 = a_1 $, we can see that $ t_1^{-1} \ A_0 \ t_1 = A_1 $. Similarly, $ t_2^{-1} \ A_0 \ t_2 = A_2 $. In other words, $A_0, A_1, A_2$ are conjugate subgroups of each other.

Now, once we know that a subgroup is a normal subgroup, in what situations is this useful? For example, when observing microstructures (twins and antiphase structures) associated with actual phase transitions, it would probably be difficult to determine “this is the result of a change to a normal subgroup.” To be frank, for crystallographers whose work centers on experiments and observation, this is not a concept that provides an immediate remedy. Normal subgroups show their power when constructing quotient groups from cosets, as described below. This is because when a normal subgroup is used as the modulus, the left cosets and right cosets become equal. It is useful when considering what kind of group remains after subtracting certain symmetry operations from the original point group or space group.

Cosets and Quotient Groups

A coset (also called a residue class) is the result of dividing (in a sense) a group by one of its subgroups and classifying the elements. If that subgroup is a normal subgroup, the cosets themselves form a group, which is called a quotient group (also known as a factor group). This brief explanation alone may not be sufficient, so let us explain more carefully below.

Let $G$ be a group and $H$ its normal subgroup. Choose two elements $g_1$ and $g_2$ from $G$, compute $g_1^{-1} g_2$, and if it coincides with some element of the normal subgroup $H$ (say $h_n$), let us define² “$g_1$ and $g_2$ belong to the same class (equivalence relation).”

Here, since the relation $g_1^{-1} g_2 = h_n \Leftrightarrow g_2 = g_1 h_n$ holds, the companions of $g_1$ must be obtained by multiplying $g_1$ by one of $h_1, h_2, h_3 \cdots$. Writing this set as $g_1 H$, we can express it as $$ g_1 H =\{g_1 h \ | \ h \in H\}$$In group theory, this $g_1 H$ is called “the coset containing $g_1$”³.

Let us look at a concrete example. Let $G$ be the set (group) of all integers under addition, and $H$ the set (subgroup) of multiples of 3. The coset containing $0$ is of course $H$ itself: $$ 0 H = \{0+h \ | \ h \in H\}$$

What about the coset containing 1? Since $ 1 H =\{1 + h \ | \ h \in H\}$, it is the set of integers that leave a remainder of 1 when divided by 3. Similarly, the coset containing 2 is the set of integers that leave a remainder of 2 when divided by 3: $ 2 H =\{2 + h \ | \ h \in H\}$. In this way, the set (group) of all integers $G$ can be divided into three cosets: $ 0 H (=H), 1 H, 2 H$.

Among the three cosets, only $0 H =(H)$ has the structure of a group, and this is the normal subgroup. On the other hand, $1 H$ and $2 H$ are not groups⁴. However, there is an important relationship among $0 H, 1 H, 2 H$. In fact, the relationships among cosets classified using a normal subgroup always satisfy the group axioms. In this case, if we define the operation between cosets as “the addition of elements arbitrarily chosen one from each coset,” then the set of three cosets $ 0 H, 1 H, 2 H$ forms a group. The identity element is, of course, $0 H$.

In group theory, a group formed by treating each such coset as an element is expressed as, for example, “the quotient group modulo the normal subgroup consisting of multiples of 3.” In notation, it is written as $G/H$. Note that while it is always possible to form cosets for any group, they do not always form a quotient group. A quotient group is formed only when the modulus group (in the preceding example, $H$) is a normal subgroup.

Cosets and Quotient Groups of Point Groups

Let us show an example of cosets using point groups. Point group $4/m$ has the following eight matrices as elements:$$
\begin{array}{l,l,l,l}
e=\begin{pmatrix}1&0&0 \\ 0&1&0 \\ 0&0&1 \end{pmatrix} ,&
s_1=\begin{pmatrix}-1&0&0 \\ 0&-1&0 \\ 0&0&1 \end{pmatrix} ,&
s_2=\begin{pmatrix}0&1&0 \\ -1&0&0 \\ 0&0&1 \end{pmatrix} ,&
s_3=\begin{pmatrix}0&-1&0 \\ 1&0&0 \\ 0&0&1 \end{pmatrix}\\
s_4=\begin{pmatrix}1&0&0 \\ 0&1&0 \\ 0&0&-1 \end{pmatrix} ,&
s_5=\begin{pmatrix}-1&0&0 \\ 0&-1&0 \\ 0&0&-1 \end{pmatrix} ,&
s_6=\begin{pmatrix}0&1&0 \\ -1&0&0 \\ 0&0&-1 \end{pmatrix} ,&
s_7=\begin{pmatrix}0&-1&0 \\ 1&0&0 \\ 0&0&-1 \end{pmatrix}
\end{array}$$

Selecting $e$ and $s_1$ from these gives a normal subgroup (point group $2$). Let us call this subgroup $eH$. Multiplying the elements of $eH$ by, say, $s_2$ gives $s_2 e = s_2, s_2 s_1 = s_3$, and multiplying by $s_3$ gives $s_3 e = s_3, s_3 s_1 = s_2$, so $s_2$ and $s_3$ belong to the same coset. Let us call this $s_2H$. Similarly, $s_4$ and $s_5$, as well as $s_6$ and $s_7$, belong to the same cosets, which we call $s_4H$ and $s_6H$, respectively. The collection of these cosets can be said to be “the quotient group modulo $eH$ (= point group $2$).”

Now, let us examine the structure of this quotient group. Between $eH$ and $s_2H$, there is a 180° rotation relationship, and the same holds between $s_4H$ and $s_6H$. Additionally, mirror operations and other relations can be found between $eH$ and $s_4H$, and between $s_2H$ and $s_6H$. As you may have already realized, the structure of this quotient group is identical to that of point group $2/m$. In essence, “when point group $2$, a subgroup of point group $4/m$, is used as the modulus to form a quotient group, the result has a structure equivalent to point group $2/m$.” Although lacking in rigor, a more intuitive expression would be: “removing the components of point group $2$ from point group $4/m$ leaves the properties of point group $2/m$,” or “dividing point group $4/m$ by point group $2$ yields point group $2/m$.” Since the notation for quotient groups is written as $G/H$, in this case it becomes $(4/m)/(2)$ — the division symbol fits quite naturally. Point group subgroups are discussed in more detail on the “5.2. Crystallographic Point Groups and Their Subgroups” page.

Cosets and Quotient Groups of Space Groups

Finally, let us consider cosets of space groups. Take space group $Pnnn$ as an example. This space group can be represented by the following set of eight matrices. Here, $n_1, n_2, n_3$ represent all integers. Also, $e’$ is not the identity matrix itself but a set of matrices containing the identity matrix, hence the prime symbol. The same applies to the others.
$$
e’=\begin{pmatrix}1&0&0&n_1\\ 0&1&0&n_2\\ 0&0&1&n_3\\ 0&0&0&1\end{pmatrix}, \
s’_1=\begin{pmatrix}1&0&0&\frac{1}{2}+n_1\\ 0&1&0&\frac{1}{2}+n_2\\ 0&0&\bar{1}&n_3\\ 0&0&0&1\end{pmatrix},\
s’_2=\begin{pmatrix}\bar{1}&0&0&n_1\\ 0&1&0&\frac{1}{2}+n_2\\ 0&0&1&\frac{1}{2}+n_3\\ 0&0&0&1\end{pmatrix},\
s’_3=\begin{pmatrix}1&0&0&\frac{1}{2}+n_1\\ 0&\bar{1}&0&n_2\\ 0&0&1&\frac{1}{2}+n_3\\ 0&0&0&1\end{pmatrix} \\
s’_4=\begin{pmatrix}\bar{1}&0&0&\frac{1}{2}+n_1\\ 0&1&0&n_2\\ 0&0&\bar{1}&\frac{1}{2}+n_3\\ 0&0&0&1\end{pmatrix},\
s’_5=\begin{pmatrix}
\bar{1}&0&0&\frac{1}{2}+n_1\\
0&\bar{1}&0&\frac{1}{2}+n_2\\
0&0&1&n_3\\
0&0&0&1\end{pmatrix},\
s’_6=\begin{pmatrix}1&0&0&n_1\\ 0&\bar{1}&0&\frac{1}{2}+n_2\\ 0&0&\bar{1}&\frac{1}{2}+n_3\\ 0&0&0&1\end{pmatrix} ,\
s’_7=\begin{pmatrix}\bar{1}&0&0&n_1\\ 0&\bar{1}&0&n_2\\ 0&0&\bar{1}&n_3\\ 0&0&0&1\end{pmatrix}
$$ $e’$ is the translation group of this space group and is a normal subgroup. Let us first form cosets using $e’$ as the modulus. As can be readily seen, if we pick any matrix from $s’_1$ and operate it with any matrix from $e’$, the result is always contained in $s’_1$. The same holds for $s’_2$ through $s’_7$. In other words, the eight matrix sets above were already cosets modulo $e’$ from the beginning. Since $e’$ is a normal subgroup, the eight matrix sets naturally have a group structure, with $e’$ as the identity element. For clarity, the operation in this case means “choosing one matrix from each set in all possible combinations and multiplying them.”

Let us also consider space group $Cc$. $$
e’_0=\begin{pmatrix}1&0&0&n_1\\ 0&1&0&n_2\\ 0&0&1&n_3 \\ 0&0&0&1\end{pmatrix},\ \
e’_1=\begin{pmatrix}1&0&0&n_1+\frac{1}{2}\\ 0&1&0&n_2+\frac{1}{2}\\ 0&0&1&n_3\\ 0&0&0&1\end{pmatrix},\ \
s’_2=\begin{pmatrix}1&0&0&n_1\\ 0&\bar{1}&0&n_2\\ 0&0&1&n_3+\frac{1}{2} \\ 0&0&0&1\end{pmatrix},\ \
s’_3=\begin{pmatrix}1&0&0&n_1+\frac{1}{2}\\ 0&\bar{1}&0&n_2+\frac{1}{2}\\ 0&0&1&n_3+\frac{1}{2}\\ 0&0&0&1\end{pmatrix} $$As before, let us use the translation group as the modulus. However, in this case, the translation group includes not only $e’_0$ but also $e’_1$. Together, $e’_0$ and $e’_1$ form the translation group (which we denote $E$), and it is of course a normal subgroup. We form cosets using $E$ as the modulus. If we pick any matrix from $s’_2$ and operate it with a matrix chosen from $E$, the result always coincides with one of the matrices in $s’_2$ or $s’_3$. Picking any matrix from $s’_3$ and doing the same also gives a result in $s’_2$ or $s’_3$⁵. This means $s’_2$ and $s’_3$ (the latter two) belong to the same coset, and together with $E$ (the former two), they form a quotient group of order 2.

Let us reexamine the relationships among the cosets when a space group is decomposed modulo its translation group. In both examples, if we focus on the upper-left 3×3 submatrix (denoted $A$), we notice that there is no duplication whatsoever.$$\begin{pmatrix}
A_{11} & A_{12} & A_{13} & B_x\\
A_{21} & A_{22} & A_{23} & B_y\\
A_{31} & A_{32} & A_{33} & B_z\\
0 & 0 & 0 & 1
\end{pmatrix}= \begin{pmatrix}
\begin{matrix} \\ \large{A} \\ \\ \end{matrix} & \begin{matrix} \\ \large{B} \\ \\ \end{matrix} \\
\begin{matrix}0&0&0\end{matrix} & 1
\end{pmatrix}
$$For space group $Pnnn$, there was originally no duplication in the $A$ parts, while for space group $Cc$, matrix sets with identical $A$ parts were grouped together into a single coset. When decomposing modulo the translation group, this property always emerges⁶. In the case of affine transformation matrices, the $A$ part only affects the $A$ part, so the coset relationships are ultimately equivalent to the structure of the group obtained by extracting only the $A$ parts. Recall that the $A$ part represents symmetry operations without translation (rotation, rotoinversion, inversion, and reflection). In other words, when a quotient group is formed from a space group using its translation subgroup as the modulus, it has the same algebraic structure as the corresponding point group⁷. This is precisely the group-theoretic relationship between space groups and point groups. Although somewhat imprecise, a more intuitive statement might be: “the set obtained by extracting only the $A$ parts from a space group is the point group.” In the examples above, space group $Pnnn$ corresponds to point group $mmm$, and space group $Cc$ corresponds to point group $m$.

Some Terms Related to Subgroups and Cosets

Index of a Subgroup

The concept that indicates the size of a subgroup relative to the original group is the “index.” For example, if group $G$ has a subgroup $H$ that contains half of the elements of $G$, the index is 2. For subgroups of finite groups, the index is simply “order of the original group / order of the subgroup.” However, this formula cannot be directly extended to infinite groups. More rigorously, the index of $H$ in $G$ is defined as the number of cosets of $H$ in $G$. In the examples above, decomposing $Pnnn$ modulo its translation group ($=P1$) yielded 8 cosets. That is, the index of $P1$ in $Pnnn$ is 8. Similarly, the index of $P1$ in $Cc$ is 4.

The index of $H$ in $G$ is usually denoted as $|G:H|$ or $[G:H]$.

Maximal Subgroup

A subgroup $H$ of a group $G$ is a maximal subgroup if there is no subgroup of $G$ (excluding $G$ and $H$ themselves) that is a supergroup of $H$. Saying that $H$ is a maximal subgroup of group $G$ is equivalent to saying that $G$ is a minimal supergroup of $H$.

For example, point group $2$ is a maximal subgroup of point group $4$, because there is no subgroup of point group $4$ that is a supergroup of point group $2$. In the case of point group $32$, the maximal subgroups are point group $3$ and (three differently oriented) point groups $2$. The former has index 2 and the latter have index 3, but note that the magnitude of the index is irrelevant.

For space groups, there are infinitely many maximal subgroups. For example, space group $P1$, which contains only translations, has elements

$ \begin{pmatrix}1&0&0&n_1\\ 0&1&0&n_2\\ 0&0&1&n_3 \\ 0&0&0&1\end{pmatrix}$ where $n_1, n_2,n_3$ are all integers,

and the subgroup consisting of the following elements is a maximal subgroup:

$ \begin{pmatrix}1&0&0&p n_1\\ 0&1&0&n_2\\ 0&0&1&n_3 \\ 0&0&0&1\end{pmatrix}$ where $n_1, n_2,n_3$ are all integers and $p$ is any prime number.

The index of this subgroup is $p$. Since there are infinitely many primes, there are also infinitely many maximal subgroups of $P1$. The concept of space group subgroups is discussed in more detail on the “5.3. Subgroups of Space Groups” page.

Footnotes

The term “conjugate” frequently appears when studying linear algebra. When we have elements \$A, B\$, what meaning does the quantity \$A^{-1} B A\$ have? Let’s think of \$A, B\$ as matrices that transform some coordinates (\$X\$) concretely. We can think of \$A^{-1} B A X\$ as the result obtained by: ① first calculating \$X’=A X\$, ② next calculating \$X” = B X’\$, and ③ finally calculating \$A^{-1} X”\$. This series of operations can also be interpreted as: ①’ transforming to another coordinate system by \$A\$, ②’ applying \$B\$ after that, and ③’ returning to the original coordinate system by \$A^{-1}\$. In other words, the \$A^{-1}\$ and \$A\$ bracketing on both sides have the role of “transforming the coordinate system of the action \$B\$,” and thinking of it this way makes sense. For example, if \$A\$ is a -90° rotation operation coinciding with the \$X\$ axis, and \$B, C\$ are 90° rotation operations coinciding with the \$Y, Z\$ axes respectively, then $$A=\begin{pmatrix}1&0&0\\0&0&1\\0&-1&0\end{pmatrix}, \\ B=\begin{pmatrix}0&0&1\\0&1&0\\-1&0&0\end{pmatrix}, \\ C =\begin{pmatrix}0&-1&0\\1&0&0\\0&0&1\end{pmatrix}= A^{-1} B A$$you can find such a relationship, and so we say “\$B\$ and \$C\$ (\$=A^{-1}BA\$) are conjugate (related) under \$A\$ (as a coordinate transformation).”
Moreover, as a digression, in the world of complex numbers, for arbitrary real numbers \$a, b\$, we say that \$a+ bi\$ and \$a-bi\$ are conjugate. This means they are conjugate with respect to a coordinate transformation that inverts the imaginary number on the complex plane. The operation can be addition or multiplication. Let’s actually verify this. For a complex number (call it \$c+ di\$), we invert its imaginary part (becoming \$c-di\$), then consider cases where the operation is addition and multiplication.
• Addition case: When we add \$a+bi\$, we get \$a+c+(b-d)i\$, and when we invert the imaginary part again, we get \$a+c-(b-d)i\$. This is equal to adding \$a-bi\$ and \$c+ di\$.
• Multiplication case: When we multiply by \$a+bi\$, we get \$ac-bd-(ad-bc)i\$, and when we invert the imaginary part again, we get \$ac-bd+(ad-bc)i\$. This is equal to multiplying \$a-bi\$ and \$c+ di\$.
Therefore, whether the operation is addition or multiplication, \$a+ bi\$ and \$a-bi\$ are conjugate with respect to the imaginary inversion operation. ↩︎
Let’s explain the equivalence relation more rigorously. When some method of comparison is defined between two elements, and we say that \$x\$ and \$y\$ are equal under that method by writing \$x\\sim y\$. When this relationship satisfies three conditions: ① \$x\\sim x\$ (reflexivity), ② if \$x\\sim y\$ then \$y\\sim x\$ (symmetry), and ③ if \$x\\sim y\$ and \$y\\sim z\$ then \$x\\sim z\$ (transitivity), then \$\\sim\$ is called an equivalence relation. For example, in the case of a comparison method of “comparing remainders when divided by 3” (i.e., mod 3 world), we can write \$4\\sim1\$ or \$16\\sim-2\$. The equals sign \$=\$ that we have used since elementary school is also a symbol representing an equivalence relationship between expressions and numbers. ↩︎
To be precise, this is a left coset. When a normal subgroup is chosen for H, the left and right cosets are equal (\$g H = H g\$), so there is no need to distinguish left and right. ↩︎
To begin with, \$1H\$ and \$2H\$ do not contain the identity element, and elements like 1 and 4 are in \$1H\$, but \$1+4=5\$ is in \$2 H\$, so the operation does not hold. ↩︎
Since $E$ is a normal subgroup, it is not strictly necessary to verify for $s’_3$, but we have written it out redundantly for clarity. ↩︎
Suppose that when decomposing a space group modulo its translation group, two cosets with identical $A$ parts were produced. If the $A$ parts are identical, their relationship is a pure translation, so the operation transforming one into the other must be contained in the translation group. However, in the quotient group modulo the translation group, the translation group acts as the identity element. The identity element cannot transform one coset into a different coset. Therefore, the initial assumption is wrong, and duplication in the $A$ parts cannot occur. ↩︎
When a space group contains neither screw operations nor glide operations, the point group can be considered as an actual subgroup rather than merely as a quotient group. ↩︎