Asynchronous SVG generation

I’m developing an editor for a typesetting language (like LaTeX) with Iced. The screen consists of a toolbar, a text editing area and a preview area. When the “preview” button (or Ctrl + S) is pressed, the text is compiled into the typesetting language, which returns a list of SVGs. In the view() method, SVG widgets are then created.

pub struct Editing {
    tool_bar: ToolBar,
    current_buffer: Buffer,
    buffers: HashMap<FileId, Buffer>,
    current_dir: PathBuf,
    preview: Preview,
    typst: TideWorld,
    auto_pairs: HashMap<char, char>,
    split_at: f32,
}

impl Editing {
    pub fn new(config: EditorConfig, current_dir: PathBuf) -> Self {
        Self {
            tool_bar: ToolBar::new(true),
            current_buffer: Buffer::new(),
            buffers: HashMap::new(),
            current_dir: current_dir.clone(),
            preview: Preview::new(),
            typst: init_world(),
            auto_pairs: config.auto_pairs,
            split_at: 800.0,
        }
    }

    fn add_buffer(&mut self, file_id: FileId, source: Source) {
        let buffer = Buffer::from_content(Content::with_text(source.text()));
        self.buffers.insert(
            file_id,
            buffer
        );
    }

    pub fn view(&self) -> Element<Message> {
        let tool_bar = self.tool_bar.view().map(Message::ToolBar);
        let editor = TextEditor::new(&self.current_buffer.content)
            .on_action(Message::ActionPerformed)
            .placeholder("Insert text here or open a new file")
            .key_binding(|key_press| {
                if let Some(ref text) = key_press.text {
                    if let Some(actual_char) = text.chars().nth(0) {
                        if self.auto_pairs.contains_key(&actual_char) {
                            if let Some(selection) = self.current_buffer.content.selection() {
                                let mut seq: Vec<Binding<Message>> = Vec::new();
                                seq.push(Binding::Insert(actual_char));
                                for c in selection.chars() {
                                    seq.push(Binding::Insert(c));
                                }
                                seq.push(Binding::Insert(
                                    *self.auto_pairs.get(&actual_char).unwrap(),
                                ));
                                Some(Binding::Sequence(seq))
                            } else {
                                Some(Binding::Sequence(vec![
                                    Binding::Insert(actual_char),
                                    Binding::Insert(*self.auto_pairs.get(&actual_char).unwrap()),
                                    Binding::Move(Motion::Left),
                                ]))
                            }
                        } else {
                            bindings(key_press)
                        }
                    } else {
                        bindings(key_press)
                    }
                } else {
                    bindings(key_press)
                }
            })
            .wrapping(iced::widget::text::Wrapping::WordOrGlyph)
            .height(Fill);
        if let Some(preview_pages) = &self.preview.content {
            let mut svg_pages = vec![];
            for page in preview_pages {
                svg_pages.push(svg(Handle::from_memory(page.clone().into_bytes())).into());
            }
          let preview = Scrollable::new(Column::with_children(svg_pages).spacing(15).padding(15))
              .width(Fill)
              .height(Fill);
            let main_screen = VSplit::new(editor, preview)
                .strategy(crate::widget::vsplit::Strategy::Left)
                .split_at(self.split_at)
                .on_resize(Message::ResizePreview);
            column![tool_bar, main_screen].into()
        } else {
            column![tool_bar, editor].into()
        }
    }

    pub fn update(&mut self, message: Message) -> Task<Message> {
        match message {
            Message::ResizePreview(split_at) => {
                self.split_at = split_at.clamp(200.0, 1500.0);
                Task::none()
            }
            Message::ActionPerformed(action) => {
                self.current_buffer.content.perform(action);
                Task::none()
            }
            Message::SvgGenerated(result) => {
                println!("async: SVG generated");
                match result {
                    Ok(svg) => self.preview.content = Some(svg),
                    Err(err) => println!("Error while generating SVG: {:?}", err),
                }
                Task::none()
            }
            Message::ToolBar(message) => {
                match message {
                    toolbar::Message::ForcePreview => {
                        self.typst
                            .reload_from_content(self.current_buffer.content.text().as_str());
                        Task::perform(
                            preview_svg(self.typst.clone()),
                            |result| Message::SvgGenerated(result))
                    },
                    toolbar::Message::FileImported(result) => {
                        match result {
                            Ok(path) => {
                                println!("file imported: {:?}", path);
                            },
                            Err(error) => {
                                println!("import error: {:?}", error);
                            }
                        }
                        Task::none()
                    }
                    toolbar::Message::OpenFile => {
                        let path = load_file_dialog("*", &["png", "jpg", "jpeg", "typ", "svg"]);
                        if let Some(import_path) = path {
                            let result = load_file(&import_path);
                            if let Ok(imported_file) = result {
                                match imported_file {
                                    ImportedFile::Asset { file_id, bytes } => {
                                        println!("imported asset: {:?}", file_id);
                                        self.typst.add_asset(file_id, bytes);
                                    }
                                    ImportedFile::TypstSource { file_id, source } => {
                                        println!("imported source file: {:?}", file_id);
                                        self.add_buffer(file_id, source.clone());
                                        if let Some(buffer) = self.buffers.get(&file_id) {
                                            self.current_buffer = buffer.clone();
                                        }
                                        self.typst.add_source(file_id, source);
                                    }
                                }
                            }
                            Task::done(Message::ToolBar(toolbar::Message::FileImported(Ok(import_path))))
                        } else {
                            Task::none()
                        }
                    }
                    toolbar::Message::Export(export_type) => match export_type {
                        ExportType::PDF => {
                            self.typst
                                .reload_from_content(self.current_buffer.content.text().as_str());
                            let path = save_file_dialog("pdf", &["pdf"]);
                            if let Some(export_path) = path {
                                Task::perform(export_pdf(self.typst.clone(), export_path, PdfOptions::default()),
                                |result| Message::ToolBar(toolbar::Message::ProjectExported(result)))
                            } else {
                                Task::none()
                            }
                        }
                        ExportType::SVG => {
                            self.typst
                                .reload_from_content(self.current_buffer.content.text().as_str());
                            let path = save_file_dialog("svg", &["svg"]);
                            if let Some(export_path) = path {
                                Task::perform(export_svg(self.typst.clone(), export_path),
                                |result| Message::ToolBar(toolbar::Message::ProjectExported(result)))
                            } else {
                                Task::none()
                            }
                        }
                        ExportType::Template => Task::none(),
                    }
                    toolbar::Message::ProjectExported(result) => {
                        match result {
                            Ok(path) => { println!("project exported at {:?}", path); }
                            Err(error) => { println!("project not exported: {:?}", error); }
                        }
                        Task::none()
                    },
                    _ => Task::none(),
                }
            }
            _ => Task::none(),
        }
    }
}

The problem with this (synchronous) method is that if the SVGs are complex, the application freezes or even crashes (indicating that it has run out of memory). I’ve managed to make compilation (and export) asynchronous using Task::perform(). But I can’t get the generation of SVG widgets not to block the UI if their creation takes several seconds.

Do you have any idea how to do this?
I think it’s this part of the code that blocks the UI if there are too many complex SVGs:

let mut svg_pages = vec![];
            for page in preview_pages { svg_pages.push(svg(Handle::from_memory(page.clone().into_bytes())).into());
}

let preview = Scrollable::new(Column::with_children(svg_pages).spacing(15).padding(15))
.width(Fill)
.height(Fill);

Thank you in advance for your help.

Have you tried to create the svg Handle in the task and not in the update method?

Yes I tried to save a Vec<Handle> by creating it with an asynchronous function in a Task::perform(). It didn’t change much. Apparently it’s calling svg() in the view() method that is costly in memory and blocks the UI.

Have you checked with the debug mode, where exactly it blocks (update, view, layout or draw)?


Here’s what I got before the application panicked with

wgpu error: Validation Error

Caused by:
    In Device::create_texture
      note: label = `iced_wgpu::image texture atlas`
    Not enough memory left.

If I do a lorem(100), the application manages fine, but here with 10000 I can no longer resize the application without it freezing or crashing.

view gets called very, very frequently, and my guess is that creating a Handle::from_memory with a page.clone() every single time is eating up all your memory

Yes, I’m afraid that’s the problem. It would explain why the application only crashes when I try to resize my VSplit, which causes a significant number of calls to view() and recreates the SVGs each time. I tried to create some kind of cache with a Vec<Svg>, but without success.

Do you have an idea that would prevent the svg(Handle::from_memory()) from being recreated each time view() is called? Or at least one that doesn’t block the UI while this operation is performed asynchronously?

you could try moving your Handle to inside your Preview struct (I don’t know what that looks like) so that you can just borrow it in fn view instead of creating the handle during view

But isn’t svg() the most expensive operation? I remember testing something similar (keeping an instance of Handle in memory), and the application always froze.

“Expensive” can mean different things. The problem here is not that svg() is slow (which it shouldn’t be, by the way), but that you’re running out of memory. That is because you are creating a Handle for each page in memory, and because each page is being cloned, you are not reusing Handles at all, just spamming them infinitely as you resize the viewer until you run out of memory.

As a rule of thumb, you don’t want to create any data in fn view. Do that in fn update and keep it in your app state.

I tried adding the Handle to my Preview structure and generating them asynchronously.

pub fn view(&self) -> Element<Message> {
        //...
        if let Some(svg_handles) = &self.preview.handle {
            let mut svg_pages = vec![];
            for page in svg_handles.to_owned() {
                svg_pages.push(svg(page).into());
            }
            let preview = Scrollable::new(Column::with_children(svg_pages).spacing(15).padding(15))
                .width(Fill)
                .height(Fill);
            let main_screen = VSplit::new(editor, preview)
                .strategy(crate::widget::vsplit::Strategy::Left)
                .split_at(self.split_at)
                .on_resize(Message::ResizePreview);
            column![tool_bar, main_screen].into()
        } else {
            column![tool_bar, editor].into()
        }
    }

    pub fn update(&mut self, message: Message) -> Task<Message> {
        match message {
            Message::ResizePreview(split_at) => {
                self.split_at = split_at.clamp(200.0, 1500.0);
                Task::none()
            }
            Message::ActionPerformed(action) => {
                self.current_buffer.content.perform(action);
                Task::none()
            }
            Message::SvgGenerated(result) => {
                println!("async: SVG generated");
                match result {
                    Ok(svg) => return Task::perform(async move {
                        let mut svg_handles: Vec<Handle> = vec![];
                        for content in svg {
                            svg_handles.push(Handle::from_memory(content.into_bytes()));
                        }
                        svg_handles
                    },
                    |handles| Message::PreviewLoaded(handles)
                    ),
                    Err(err) => println!("Error while generating SVG: {:?}", err),
                }
                Task::none()
            }
            Message::PreviewLoaded(svg_handles) => {
                println!("async: preview loaded");
                self.preview.handle = Some(svg_handles);
                Task::none()
            }
            Message::ToolBar(message) => {
                match message {
                    toolbar::Message::ForcePreview => {
                        self.typst.reload_from_content(self.current_buffer.content.text().as_str());
                        Task::perform(
                            preview_svg(self.typst.clone()),
                            |result| Message::SvgGenerated(result))
                    },
                    //...
                    _ => Task::none(),
                }
            }
            _ => Task::none(),
        }
    }
}

And Preview:

use iced::advanced::svg::Handle;

pub struct Preview {
    pub handle: Option<Vec<Handle>>,
    pub is_inverted: bool,
}

impl Preview {
    pub fn new() -> Self {
        Self {
            handle: None,
            is_inverted: false,
        }
    }
}

The problem remains the same, the application crashes because of memory. I know that generating data in view() is bad practice, but how am I supposed to call svg() anywhere other than in this method?

do you have this up in a repo somewhere?

Nope, but I can provide you with anything you want. I also have a video showing how to reproduce the crash and when the application freezes.

I tried replicating it here generating 500 SVGs and yeah, it does seem sluggish and it’s not the text_editor that is slow.

I could be wrong, but my inclination would be to try my hand at a custom “viewer” widget to be smarter about having many svgs and reusing them instead of just shoving hundreds of svg() into a column.

But if anyone else has the answer, by all means please chime in.

I share an interesting fact: if the SVG is not resized, the application is fluid, even if it is recreated in view(). On the other hand, as soon as it is enlarged or shrunk, the application starts to freeze (and it can crash if there are a lot of SVGs generated). Here’s a video illustrating this (am sharing via WeTransfer because you can’t upload a video here): Unique Download Link | WeTransfer

Anyway, thanks for your help :slight_smile:

1 Like

The plot thickens! That’s interesting… It behaves a little different here. Since I don’t have a resizer between the editor and viewer, I was testing by resizing the window.

I do experience sluggishness during window resizing operations but not after. Scrolling feels smooth even with 100 pages regardless of window size. It’s just during the resizing that things feel slow. The behavior is identical on tiny_skia or wgpu.

My toy code here for others to experiment with if they want to help debug. You’ll obviously need iced with the svg feature enabled and reqwest for the lipsum generation.

use iced::widget::{column, *};
use iced::{Alignment::*, Center, Element, Fill, Task};
use svg::Handle;

pub fn main() -> iced::Result {
    iced::run(
        "iced • svgs typesetting",
        SvgViewer::update,
        SvgViewer::view,
    )
}

#[derive(Debug, Default)]
struct SvgViewer {
    content: text_editor::Content,
    previews: Option<Vec<Handle>>,
    is_loading_text: bool,
}

#[derive(Debug, Clone)]
pub enum Message {
    Editor(text_editor::Action),
    UpdatedPreviews(Vec<String>),
    LoadLoremIpsum,
    LoremIpsumLoaded(String),
}

async fn generate_svg_previews(text: String) -> Vec<String> {
    let svg_width = 380;
    let svg_height = 300;
    let chars_per_line = 40;
    let lines_per_page = 12;

    let mut all_lines = Vec::new();

    for line in text.lines() {
        if line.trim().is_empty() {
            // non-breaking space for empty lines
            all_lines.push(String::from("\u{00A0}"));
            continue;
        }

        let mut start = 0;
        let chars: Vec<char> = line.chars().collect();

        while start < chars.len() {
            let end = if start + chars_per_line >= chars.len() {
                chars.len()
            } else {
                // find last whitespace within limit
                let mut pos = start + chars_per_line;
                while pos > start && !chars[pos].is_whitespace() {
                    pos -= 1;
                }
                if pos == start {
                    // no whitespace found, hard break
                    start + chars_per_line
                } else {
                    pos + 1 // include the whitespace
                }
            };

            all_lines.push(chars[start..end].iter().collect());
            start = end;
        }
    }

    // create SVGs
    let mut svgs = Vec::new();
    let page_count = (all_lines.len() + lines_per_page - 1) / lines_per_page;
    svgs.reserve(page_count);

    for chunk in all_lines.chunks(lines_per_page) {
        let mut tspans = String::with_capacity(chunk.len() * 50);

        for (i, line) in chunk.iter().enumerate() {
            tspans.push_str(&format!(
                "<tspan x=\"20\" dy=\"{}\">{}</tspan>",
                if i == 0 { "0" } else { "20" },
                line
            ));
        }

        svgs.push(format!(
            r##"<svg xmlns="http://www.w3.org/2000/svg" width="{}" height="{}" viewBox="0 0 {} {}">
                <rect width="100%" height="100%" fill="white" stroke="gray" stroke-width="1"/>
                <text x="20" y="30" font-family="monospace" font-size="14" fill="black">
                    {}
                </text>
            </svg>"##,
            svg_width, svg_height, svg_width, svg_height, tspans
        ));
    }

    svgs
}

async fn fetch_lorem_ipsum() -> String {
    let default_text = "\
        Lorem ipsum dolor sit amet, consectetur adipiscing elit.\n\n\
        Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.\n\n\
        Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris.";

    let client = reqwest::Client::new();
    match client
        .get("https://baconipsum.com/api/?type=all-meat&paras=100")
        .send()
        .await
    {
        Ok(response) => {
            if let Ok(text) = response.text().await {
                // bracket stripping and splitting
                let text = text.trim();
                if text.starts_with('[') && text.ends_with(']') {
                    let content = &text[1..text.len() - 1]; // strip outer brackets
                    return content
                        .split("\",\"") // split on delimiter between JSON strings
                        .map(|s| s.trim_matches(|c| c == '"' || c == '\\')) // remove quotes
                        .collect::<Vec<_>>()
                        .join("\n\n");
                }
            }
            default_text.to_string()
        }
        Err(_) => default_text.to_string(),
    }
}

impl SvgViewer {
    fn update(&mut self, message: Message) -> Task<Message> {
        match message {
            Message::Editor(action) => {
                let is_edit = action.is_edit();
                self.content.perform(action);

                // if the action is an edit, trigger async SVG generation
                if is_edit {
                    let text = self.content.text();
                    Task::perform(generate_svg_previews(text), Message::UpdatedPreviews)
                } else {
                    Task::none()
                }
            }
            Message::UpdatedPreviews(svg_strings) => {
                if !svg_strings.is_empty() {
                    let handles = svg_strings
                        .into_iter()
                        .map(|s| Handle::from_memory(s.into_bytes()))
                        .collect();

                    self.previews = Some(handles);
                }

                Task::none()
            }
            Message::LoadLoremIpsum => {
                self.is_loading_text = true;
                Task::perform(fetch_lorem_ipsum(), Message::LoremIpsumLoaded)
            }
            Message::LoremIpsumLoaded(text) => {
                self.content = text_editor::Content::with_text(&text);
                self.is_loading_text = false;

                Task::perform(generate_svg_previews(text), Message::UpdatedPreviews)
            }
        }
    }

    fn view(&self) -> Element<Message> {
        let lorem_button = button(
            text(if self.is_loading_text {
                "Loading Lorem Ipsum..."
            } else {
                "Load Lorem Ipsum Text"
            })
            .size(12),
        )
        .padding(5)
        .on_press_maybe((!self.is_loading_text).then_some(Message::LoadLoremIpsum));

        let editor = column![
            row![
                text("Type text below (separate pages with empty lines):").size(16),
                horizontal_space(),
                lorem_button
            ]
            .padding(2)
            .align_y(Center),
            text_editor(&self.content)
                .on_action(Message::Editor)
                .height(Fill)
        ]
        .spacing(10)
        .padding(10)
        .width(Fill);

        let preview_panel = if let Some(handles) = &self.previews {
            let svg_pages = handles.iter().enumerate().map(|(i, handle)| {
                row![
                    container(text(i + 1).align_x(End).size(12)).align_right(20),
                    svg(handle.clone())
                ]
                .align_y(End)
                .into()
            });

            scrollable(column(svg_pages).spacing(15).padding(15))
                .width(Fill)
                .height(Fill)
                .into()
        } else {
            Element::from(center(text("Type something to see SVG previews").size(16)))
        };

        row![editor, preview_panel].height(Fill).into()
    }
}

Well, it’s not totally different, since it does the same thing to me (in fact, as soon as the SVG is resized, the application freezes or crashes). Maybe that’s what the documentation means when it says Svg images can have a considerable rendering cost when resized, specially when they are complex. (Svg in iced::widget::svg - Rust).

Couldn’t we store a Vec<Svg> to avoid having to call svg() every time view() is called and reuse the widgets cached in the Vec<>?

Svg will need to call fn layout and fn draw when it gets resized, and one of these must be what’s costly, so holding it in your state won’t help. It’s no so much fn new but what happens shortly thereafter, if that makes sense. (Also holding Element<'a>s in your state is never really doable)

I don’t know which part of those function is particularly costly, but that’s my working hypothesis.

Maybe you could get clever by storing the last computed layouts in fn layout and reuse them until (pick a number) ~200ms have passed since the last call to fn layout, at which point you do recompute everything, such that you’re not spamming this function repeatedly. Naturally that would also require storing the Instant of the last call to fn layout

Separately you could also implement a windowing viewer column that uses pop() to add/remove the svgs from view, replacing them with empty containers with the same size as the svgs when they are out of view (“easy” in your case since you know the page size in advance) so that you’re only ever resizing the svgs of the 2-3 pages that are visible.

Perhaps suggestion #2 is easier to try than #1

Thanks for these suggestions. Unfortunately, the second one surely won’t solve the problem as I’m able to reproduce the same UI crashes/freezes with only one SVG generated (I just need to resize the window or my VSplit to get my application buggy). So I don’t think the number of SVGs is the source of the error. I’ll try your first suggestion but I can’t guarantee anything as I’m not an Iced expert.

In the meantime, I’ll probably open an issue or talk about it on the Discord server.