使用Matlab实现基于MATLAB交互式粒度卷积的合成器

Coursework Assessment Pro-forma
Module Code: CM3106
Module Title: Multimedia
Lecturer: Prof. David Marshall
Assessment Title: MATLAB Interactive Granular Convolution-based Synthesiser
Assessment Number: 1
Date Set: Tuesday October 15, 2019
Submission Date and Time: Friday December 13, 2019 at 9:30am
Return Date: Within 3 weeks of submission date and feedback at Revision Lecture Week 12
This assignment is worth 30 % of the total marks available for this module. If coursework is
submitted late (and where there are no extenuating circumstances):
1 If the assessment is submitted no later than 24 hours after the deadline,
the mark for the assessment will be capped at the minimum pass mark;
2 If the assessment is submitted more than 24 hours after the deadline, a
mark of 0 will be given for the assessment.
Your submission must include the official Coursework Submission Cover sheet, which can be
found here:
https://docs.cs.cf.ac.uk/downloads/coursework/Coversheet.pdf
Submission Instructions
The module leader clearly defines how the assessment should be submitted. For example,
should students submit a single PDF or multiple files. The coursework specification will
specify the number of files expected along with the permitted filetypes.
Description Type Name
Cover sheet Compulsory One PDF (.pdf) file [student number].pdf
Q1 Compulsory One PDF (.pdf) or Word file (.doc or .docx) Q1_[student number].pdf/doc/docx
Compulsory One or more program source file and supporting data No restriction
Any code submitted will be run on a Macintosh PC (MATLAB 2019a) and must be submitted
as stipulated in the instructions above.
Any deviation from the submission instructions above (including the number and types of
files submitted) will result in a mark of zero for the assessment or question part.
Staff reserve the right to invite students to a meeting to discuss coursework submissions
Assignment
The exercise involves developing a MATLAB Interactive Granular Convolution -based Synthesiser.
You must submit a typeset report in PDF format — a short (about 1500 words) written description
conveying all the appropriate information to demonstrate and explain your programming philosophy
is all that is required. You should also submit all MATLAB code as text files which should be zipped
into a single file for submission. For complete details see the following sheets.
See below for complete details
Learning Outcomes Assessed
1. Demonstrate an awareness of the factors involved in multimedia data processing and
analysis through detailed study of potential problems of processing multimedia data and of
implications of multimedia formats and types.
2. Show familiarity, including practical investigation, with a range of multimedia formats for
text, graphics, animation, audio, images and video by detailed study of common multimedia
formats.
4. Possess an understanding of the underlying concepts, representation and processing of
multimedia data with relevance to the applications in digital audio, imagery and video,
including digital audio processing and synthesis, audio/image/video compression and
multimedia data retrieval.
Criteria for assessment
Credit will be awarded against the following criteria.
Specifically the marks will be apportioned as follows:
• 10 marks are available for the basic demonstration of satisfaction of the coursework
requirements – overview of solution, appropriate satisfaction of all the basic criteria for
the exercise (specified below). This is via a demo at the last lab class and also the written
report.
• 30 marks are available a system that clearly demonstrates a working functional system
with suitable MATLAB implementations of basic exercise criteria.
• 30 marks are available for a clear write up. The write up should clearly state your design
and implementation strategy.
• 30 marks are available for your design and incorporation of other features and media
beyond the basic coursework specification.
Feedback and suggestion for future learning
Feedback on your coursework will address the above criteria.
The Individual project work will be returned in a scheduled revision lecture (TBA) during
reading week (Week 12, Semester 1).
Individual feedback will be given via Grade Centre.
Further Oral feedback will be provided in the revision lecture where the Individual project
work is returned.
Multimedia Module No: CM3106
Assessed Coursework:
MATLAB Interactive Granular Convolutionbased Synthesiser
Submission of Coursework
• Hand in Date: Submission of all material (Short Report and MATLAB code) by 9.30AM on Friday 13th December (Week 11, Autumn
Semester) via Learning Central.
• You must also get your coursework ‘signed off ’ by the course tutor/lecturer to verify to what extent the programs work according to
specification.
You must therefore demonstrate your working solution to the
tutor in the Week 11 Lab Classes
• The tutor/lecturer is only guaranteed to be available to sign at CM3106
laboratory sessions.
– The lab tutor will make comments on your demonstration solution
via Grade Centre.
– The lab tutor plays no part in assigning any of the marks for this
coursework
• This work is the only coursework component, i.e. This piece of work
is worth 30% of the marks for the whole module.
Submission Details
The exercise involves developing an Interactive Granular Convolution-based
Synthesiser in MATLAB. You must submit a typeset report in PDF format
plus all MATLAB code. Your coursework submission to Grade Centre
must include:
• A cover page that details the following:
1
– Student number, student name, module code, module title, coursework title, lecturer, hours spent on this exercise, and special provision (if applicable).
• A report of about 1500 words (approximately 3–4 pages of text), with
additional diagrams, screenshots etc., which are encouraged, typeset in
PDF format, detailing the following:
– An overview of your program design and implementation.
– A basic algorithmic description of the main elements of your solution and how they satisfy the basic requirement listed below.
– You should highlight any novel features or those above the basic
requirements.
• In addition to the short report, a copy of all code together with all the
assets required to run the code (such as sound files) should be provided.
– Include all code as text files.
– Make sure that running the program does not require any additional setup (such as configuring paths).
– All code files including all support media assets should be submitted as single zip file collection.
– Ensure your name and student ID is associated with each file.
• Ensure that your student number and name prominently appear in
each file that makes up your submission.
2
Assessed Coursework
You should develop an Interactive Granular Convolution-based Synthesiser with graphical filtering capabilities. It is highly recommended
that you implement this coursework in MATLAB, as many examples discussed in lectures, labs etc. use MATLAB code that you are perfectly free to
use (with suitable referencing) and adapt as part of your solution. However,
implementations in other languages are acceptable.
The inspiration for the work is a piece of audio software called Iris by
Izotope Inc and a piece of hardware called a Granular Convolver
3
The Granular Convolver device essentially allows you to record sounds
and slice them into small fragments referred to as grains — see lecture notes
on “Granular Synthesis”. Once a sound has been recorded, you can convolve
the grains (in any configuration you choose) with any new input you care to
send into the device — the original device used live input, you only need to
apply it to other (pre-recorded) input sound. Convolving of the two signals
creates completely new and interesting sounds. See this youtube video for
more details.
Iris is an innovative sampling Fourier transform-based re-synthesiser. You
can input up to 3 waveforms and dissect and process them in many ways.
Using Iris’s spectrogram display and easy drawing/selection tools to spotlight the most interesting spectral characteristics you can blend and layer
your modified samples with some filters that are unrealisable by standard
filter designs. The sounds can then subsequently processed with other audio
effects.
Useful web links:
• Izotope Iris main web page:
https://www.izotope.com/en/products/create-and-design/iris.html
• Izotope Iris Help:
http://help.izotope.com/docs/izotope-iris-help.pdf
• Izotope Iris Video Tutorials: Izotope Video Guides/Tutorials
4
• A 10 day trial demo version of Iris is freely available:
Iris download page here
Please note that you are only required to create a program that emulates the basic spectrogram editing and playback functionality of Iris and
implements some form of granular convolver with some additional additional
audio processing described in more detail below.
Coursework Requirements
The following basic requirements should be met in order to gain average to
good marks:
• You should implement a method of reading in an input audio file,
compute its underlying time-frequency distribution (i.e. short-term
Fourier transform) and provide an interactive means of filtering this
audio via the editing of its displayed spectrogram.
• The resulting edited audio then needs to be played back: Playback
need only be monophonic, i.e. it need only be able to play one note
of audio at a time. Playback should be able to control:
– the pitch of the audio — the audio pitch should be able to be
playback at a given new pitch but at the same tempo.
– the tempo of the audio — the audio tempo should be able to be
playback at a given new tempo but preserving the same pitch.
• Implement a basic form of Granular Convolution:
– If the short-term Fourier transform is computed on a very small
sample window then aspects of granular synthesis can be implemented.
– A basic description of short-term granular synthesis is discussed
in the lectures. However, you are expected to read around and
develop a more in-depth and thorough understanding of granular
synthesis as part of this coursework exercise.
∗ The lecture and tutorial notes reference some basic Granular
Synthesis code (which you could build upon and enhance) and
also demo some Granular synthesisers (publicly and commercially available)
5
∗ Another example of a commercial Granular Synthesiser is:
http://www.newsonicarts.com/html/granite.php
∗ The lecture notes also reference some basic applications of
convolution that you can build upon.
∗ Applying one single grain as a convolver is the basic requirement, although the potential to include more in a multitude
of configurations is encourages as and additional feature (see
below).
• In addition you should provide some additional audio processing:
– You should implement some form of volume shaping or envelope
shaping to further control or modulate the basic sounds produced.
• The playback of audio DOES NOT have to operate in real
time – you can read the data in, generate/process the audio and then
play the output, much like the demo MATLAB programs in the lectures/tutorials.
• Your basic system need not have an advanced GUI (nothing as elaborate as Iris is expected!). You could satisfy all the requirements with a
minimal GUI to display the audio/spectrogram and allow basic interactive editing.
– You may use any GUI development environment to create your
basic GUI.
– MATLAB’s App Designer (or MATLAB’s older GUIDE environment) is recommended.
• Your synthesis/effects pipeline for generating sounds to output can be
fixed, i.e. the order of processing elements can be a series of processes
that are hard wired in the MATLAB code: You may filter before or
after convolving, envelope shaping before or after either these.
In order to gain higher marks you need to think of adding TWO novel
extensions or additional features. There are endless possibilities here and
you are encouraged to think of your own extensions. Here are a few
suggestions:
• Advanced granular convolution:
6
– Applying multiple grains as convolvers in series or, even, parallel?
– Looping through a short audio sample selecting (in series or randonly) grain convolvers.
– Selection of a set number of grains as in the original granular
convolver hardware.
• Like Iris, you could provide multi-layer sample playback using more
than one audio source.
• You could provide advanced playback functionality to allow for looping
of sections of audio, reversing sections of audio etc.
• You could implement some additional digital audio effects to the output synthesised notes — You may re-use any audio code supplied in
lectures or labs to satisfy this sub-task.
• Provide a user-friendly editor for the audio and/or to enter musical
data.
– GUI elements to control the synthesis, filtering/modulation and
sound output may be provided.
• Provide support for polyphonic output.
• Provide MIDI support for data input.
• A modular synthesis/effects approach may be developed where the
sound generation and subsequent processing pipeline (order of synthesis/effects) is not static and can be configured at runtime.
Demonstrating your system
You will be required to demonstrate your final system to the lab tutor in order
to verify the extent to which the programs work according to specification.
The tutor/lecturer is only guaranteed to be available to sign at Multimedia
Laboratory Sessions.
For the demo you need only play a short number of sounds/notes. This
will be enough to demonstrate that you can make interesting atmospheric
and/or musical sounds!
If you have any additional features in your system, you should demonstrate how they work and that they function accordingly.
7
Keeping the Noise Down in the Labs
Please remember to use headphones when developing your system
in the lab: you may bring your own headphones or they can be
supplied by the tutor
Some Useful Links
You may find these useful in helping you develop your solution to the above
assessed coursework:
• http://www.jyu.fi/musica/miditoolbox/ — MATLAB MidiToolbox.
– Reads/Writes Midi files, converts midi between note number, musical notes/pitches and frequencies.
– See also http://users.cs.cf.ac.uk/Dave.Marshall/Multimedia/exercises BSC/
for local copy of the MidiToolbox.
• http://labrosa.ee.columbia.edu/matlab/ — MATLAB Audio Processing Examples
• http://www.phy.mtu.edu/ suits/notefreqs.html — musical pitches frequency relationship.
• http://en.wikipedia.org/wiki/List of meantone intervals:
List of pitch/tone intervals as ratios. (Useful for the phase vocoder —
see Tutorial 2 notes).
A quick web search should provides plenty of free audio samples for you
to use to develop and test your system. Here are a few links to get you
started:
• http://www.loopasonic.com/
• https://www.looperman.com/loops
• http://www.phatdrumloops.com
Prof. David Marshall. October 2019
8