I am opening this new section (as I must have forgotten to include it before) with this topic.
Most of us today use sample libraries. There are those who do so exclusively to improve the sound of their studio experience on digital pianos, those who use them to add color and get an acoustic feedback of their compositions, and finally those who use them for professional purposes to create promos, film music, sound design, soundtracks for video games, etc., etc.
However, few know what the knowledge and design phases are that form the basis for creating a library.
I remain of the opinion that while it is fine to use an instrument, by also knowing its construction technology, one can always draw something extra from it.
Everything I will say in this article can then be used to enable a buyer to technically evaluate the specifications of the various libraries currently available on the market, because while it is good to listen to the sound of library demos, we must not trust demos too much...
We will not delve too deeply into the implementation particularities of the player, i.e., that tool necessary for the reproduction of the sample at the moment a key is pressed or, more generally, upon reading a given MIDI input; I am much more interested in talking about the constructive particularities of the libraries, whether they are supported by samplers designed directly by the product's creator or whether they make use of existing samplers—it doesn't matter for the purposes of this article.
I specify right away that I might be very prolix, therefore I recommend reading only to those people seriously interested in the subject.
Let's begin...
The first essential piece for producing a library must be the in-depth knowledge of the "Standard MIDI" protocol specifications. Without knowing this protocol, one cannot begin to design because one has no idea of the possibilities that this set of rules offers. We will not give a lesson on MIDI since an in-depth explanation of this protocol would require pages of an entire book; we will stop to describe some key concepts...
MIDI data transfer occurs between two devices, one of which performs the role of SENDER and the other RECEIVER. It is no coincidence that peripherals with a MIDI interface carry the labels: "MIDI OUT", which is the port from which MIDI data exits, and "MIDI IN", which is the port used for receiving MIDI data. The MIDI protocol is a type of serial protocol, meaning where data follows one after another commanded by two particular bits, one for start and one for end, between which lies what is called the "MIDI Word". This is composed of two types of Bytes: an 8-bit status byte that provides information on channels, note on/off, etc., and the data byte that provides information on velocity, note pitch, etc. These bytes have an MSB (Most significant bit) that serves to distinguish which message refers to the status byte and which message refers to the data byte. Especially in the status byte, it is interesting to talk about Nibbles (sequences of 4 bits); therefore, an 8-bit status byte will be composed of two nibbles. As we have seen, the first nibble is composed of a set of 4 bits, of which the first refers to the MSB, and the other 3 refer to the type of message: Channel Message or System Message. The second Nibble, instead, provides information on the MIDI channel on which to perform the routing of the data byte's data. Being composed of 4 bits, we will have 2^4=16 addressable channels. In the data byte, for example, velocity is recorded, and a MIDI word can contain more than one data byte. Since the first bit is used to distinguish between status byte and data byte (MSB), we will have 7 bits remaining available for the encoding of the MIDI data. For example, velocities range from 0 to 127 because 7 bits guarantee 128 possible combinations: 2^7 = 128 (the value 0 is obviously the first piece of data).
In the minds of the engineers who find themselves creating a sound library, there is always the desire to create a product that is as close as possible to the acoustic one, meaning that its use within a musical context gives a result as close as possible to that obtainable from a recording of the same piece performed acoustically.
The first obstacle is represented by the choice of instrument: I am not talking, of course, about choosing between a violin, a piano, or a harp, but about the choice of the instrument among many pianos, or many violins, many harps, etc. One wants to seek out an exemplary instrument that has a better sound than the others. This instrument must then be continuously kept"} ,
under the control of a technician so that its tuning and timbral characteristics do not change. Recording the same note on two different levels with different tunings due to natural tension losses could be destructive for the creation of a library.
Coming to the act of sampling, there are insurmountable problems for which no solution exists; let's analyze some of them...
How many of us have not heard the phrase: "The whole is never equal to the sum of its parts"?
There could be no better occasion than this to bring this phrase into play.
Recording two notes separately and then layering them to make them sound together, versus playing two notes together while recording them, is not at all the same thing. If there were any doubts, I reserve the right to post a spectrum analysis to demonstrate this statement on paper, but I believe the following explanation is sufficiently convincing.
When playing a note, the fundamental frequency and its harmonics are created (this applies to any instrument).
When playing another note, the same thing happens; let's not dwell on it.
When playing two notes together, the frequencies of the two notes interact with each other due to sympathetic resonance. In particular, some frequencies will be attenuated and others reinforced; this interplay of frequencies can be clearly visible at the spectrum level but is also evident upon listening.
It will therefore be necessary, during the design phase of a library, to become aware of this problem by studying the phenomenon and finding a theoretical or empirical law that resolves it. Today, indeed, various libraries are available that feature the use of Sympathetic Resonance with the possibility of adjusting its participation via an "Amount" knob.
Let's move on to the acoustic problems during the capture phase... There are essentially two methods for sample capture:
- Instrument positioned in an anechoic chamber and recording samples as dry as possible, to then color them using software reverbs or outboard reverb hardware.
- Instrument recorded in a concert hall.
Obviously, the choice between these two types is also strongly influenced by the target audience for which this library is intended. For a general-purpose library, the first might be recommended; for a classical use library, the second choice is certainly advisable.
In the case of the first choice, there is still an additional problem to consider: namely, that environmental simulation through the use of plugins or external reverb hardware can be limiting. Let me explain better... The reverb effect is psychoacoustically recognized thanks to the reflections of the environment. This means that the direct sound produced by an instrument propagates through the medium (typically air) and then hits the walls of the environment. Normally, the sound reflected off the walls reaches the listener's ear or, in this case, the microphone capsule with a certain delay. We must say that in perceptual reality, these delays belong to the domain of real numbers (continuous), whereas in reproduction with plugins, the progression will be discrete (belonging to the set of natural numbers N). As a result, no plugin can ever psychoacustically recreate the same reverb effect existing in nature, and this certainly entails differences between the two types of capture. Additionally, another problem is linked to the absorption coefficient of the walls per frequency band. For this reason, convolution reverbs were invented, which use an integral product to recreate the acoustic conditions of the room, including its frequency response, but we must not forget that we are still in the discrete field.
Ultimately, returning to our library's target, this will also involve the choice of capture instrumentation (microphones, preamps, converters, EQ, and compressors) and the positioning of the microphones and their spot points. For a library created for Jazz purposes, it will certainly be more effective to position the microphones very close to the instrument, as Jazz is a style where sonic detail is sought. This choice would certainly not suit the classical repertoire.
For this reason, engineers thought it wise to capture the instrument from different spatial perspectives, providing the now well-known positions: CLOSE, PLAYER, ROOM, STAGE, SIDE, etc. In this way, the end user, thanks to a skillfully programmed graphical user interface (GUI), will be able to mix these perspectives to find the sound that best suits their work. However, let us remember once again the saying that the whole is not
equal to the sum of its parts !! By this I mean that this artifice can never replace a recording specifically thought out for a particular musical style.
Lastly, before capturing, it will be necessary to plan what kind of sounds to capture; this choice clearly varies based on the instrument to be sampled. For a violin, it would be appropriate to capture articulations: staccato, pizzicato, legato, spiccato, vibrato, etc., etc. For the piano: staccato, release, with sostenuto pedal, without sostenuto pedal, with soft pedal, sostenuto/repetition, hammer noises, damper noise, etc.
This choice cannot be made without the advice of a musician who must work closely with the engineer, proposing sampling solutions capable of covering all situations that may be encountered within a score.
This operation must obviously be done for all velocities.
We have therefore arrived at the choice of the number of levels. As we saw previously, the MIDI databyte relating to velocity is capable of containing values ranging from 0 to 127 levels. Creating a library with 127 levels would be costly in terms of timing, workload, and final file size. 127 levels for 88 notes involve 11,176 samples for each articulation. Considering an average calculated using the data below, of 5 MB per sample at 16-bit stereo 44100 for 30 seconds, we would have the beauty of 55 GB for a single articulation. Considering that we want to sample only 4 articulations, our library would weigh 220 GB !! At 24-bit, we would rise to 338 GB... Unfeasible.
Therefore, it was thought to create samples capable of being used within a certain velocity range and to exploit the velocity value within the sampler as the normalization volume of the sample in use within that range. In plain English, having 127 levels available, we could logarithmically divide the signal ranging from 0 dBFS to -infinity into 127 logarithmically equidistant levels so as to make the samples sound at compatible levels with each other.
To do this, once sampled, it is necessary to normalize all samples to 0 dBFS (the volumes of the samples heard by a normal audio player will all be the same, but obviously the timbre will be different; a note played softly, even if amplified to reach the 0 dB level, will certainly have fewer harmonics than a note played loudly). The differentiation is then made by mapping the samples into velocity ranges. (If I haven't been clear enough, tell me and I will try to explain things with some examples).
Clearly, the more levels a library has, the more progressive and credible its timbre will be as velocity increases, but as we have seen, we also have to deal with size !!
Libraries considered excellent hover around 18 sampling levels and up; very good are around 12; good are 9; sufficient are 7; insufficient are below 7.
In instruments like the piano, it is also possible to provide for what is known as Round Robins, which is a technique that allows for the stochastic reproduction of a sample among 2/3/4/5... available for the same note in the same velocity range during repetitions. It is a valid technique both for soft pedal samples, sostenuto pedal, without sostenuto pedal, etc., and it clearly serves to increase the credibility of a library. Repeating a string that is already vibrating indeed produces a different sound each time compared to a string in a state of rest. This happens because, obviously, one must consider where the hammer hits the vibrating string. The distance between hammer and string in the case of rest is always constant; in the case of a vibrating string, this will vary! Furthermore, especially in the first moments, we should evaluate whether the interference of the hammer on the string has constructive or destructive effects on the vibration. In particular, once the hammer hits the string, it also takes a certain amount of time to let it vibrate, so in the first moments of impact on a vibrating string, some phenomena will occur that will somehow modify the sound. What better technique for simulating this peculiarity than the use of Round Robin?
Other considerations, still staying with the piano which is the instrument closest to me, can be those regarding half-pedaling samples, which is the technique used by pianists to accelerate the envelope curve of the sounds without extinguishing them abruptly, by vibrating the pedal on the strings.
Once all the samples are obtained, they must be processed at the sound restoration level (of vital importance). This is a topic of vital importance because it could cause serious problems in the final use of the library which, if underestimated, could truly determine its"}
badly done. The law states that samples must absolutely not contain background noise. If we were to record an acoustic performance, it would have its own constant background noise, more or less audible depending on the dynamics of the recording (in very percussive parts it would certainly be barely audible, while it would be much more audible in the pianissimos; in any case, we could assume its presence as constant). What happens if I find background noise within the samples of the libraries? It turns out that the background noises add up. The result is that in a dyad I might notice little background noise, but in an 8-note chord, this would become very present to me. Obviously, in a composition, it is very easy to find both chords and dyads, and it would result in a non-homogeneous presence of background noise. For this reason, it is strictly necessary to completely eliminate the background noise. However, the elimination of background noise, even if performed spectrally, always tends to eat away at something in the high frequencies, especially on the pianissimo samples. For this reason, there is a tendency to record the samples a bit more brightly so that at the time of restoration, a sound is returned that is not "compromised" by this operation.
It is time to normalize all the samples to the same level in order to make the reasoning done above regarding velocity feasible.
Finally, the articulation groups are created and the individual samples of each group are mapped into the sampler's MIDI matrix. The code is written to take into account all the relevant functionalities, and the various plugins accompanying the library (eq, compressor, reverb, sympathetic resonance, etc.) and the graphical user interface, which must be intuitive and functional for all types of users, are implemented.
I have tried to reduce the discussion to its essence, as this is infinitely more complex, but I hope that at least I have conveyed the idea of the difficulties involved in creating a library that we all commonly use every day without knowing, however, the amount of work behind it, and that I have provided some information for those who find themselves spending money to buy a new sample library.
