Ticket #180 (new defect)

Opened 6 years ago

Last modified 6 years ago

Document3D: STEP recomposition consumes too much memory

Reported by: pcosquer Owned by:
Priority: major Milestone:
Component: document3D Version:
Keywords: step compose Cc:

Description

bug description

Summary

STEP "recompositions" consumes too much memory and CPU. Server can be overloaded if several users download a recomposed STEP at the same time.

Environment

  • Version/SVN revision: 1960
  • Operating system (server): debian squeeze

Steps to reproduce this issue

  1. Create a document3D with a big STEP file
  2. attach a part and decompose the STEP file
  3. Download the recompose STEP file -> repeat this action with several users at the same time

bug analysis

How the recomposition is done:

  1. the assembly is built in memory (only the structure, not the geometry), see Document3DController.get_product
  2. the assembly is serialized (json)
  3. a script (another process) builds the STEP file: work is done by OpenCascade?, and consumes a lot of memory
  4. the file is served.

fix recommendation

An ideal fix: recomposition does not consume memory. Maybe it is possible to rewrite the script and improves memory usage.

A temporary? fix:

  • cache the generated step file: a good file name would be an hash (sha256) of the json file appended to the document file id
  • if another user downloads the step file:
    • call get_product()
    • hash the json content
    • if file exists: serve it
  • Where should the temporary file be stored, /tmp/ ?
  • old files ?

Note: if several step files are recomposed at the same time, this hack is ineffective.


Change History

comment:1 Changed 6 years ago by Galech

bonjour Pierre! comment allez vous?

j ai revise un petit peu le script.

Peut etre vous pouvez liberer la memoire


if not link.product.label_reference:

my_step_importer = NEW_STEP_Import(link.product.doc_path)
lr_2= TDF_LabelSequence()
my_step_importer.shape_tool.GetFreeShapes?(lr_2)
add_labels(link.product,lr_2.Value(1),my_step_importer.shape_tool)
link.product.label_reference=lr_2.Value(1)

###############################
#### try to free my_step_importer ####
###############################

comment:2 Changed 6 years ago by pcosquer

In [2489/main]:

document3D: composition: add a dummy composer
TODO:

  • add location
  • remove useless/wrong lines from original step files
  • better header
  • hope it will generate at least one valid step file

see #180

Note: See TracTickets for help on using tickets.